[JENKINS-EA] Lucene-Solr-master-Linux (64bit/jdk-9-ea+132) - Build # 17730 - Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/17730/
Java: 64bit/jdk-9-ea+132 -XX:-UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  
org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.testDelegationTokenCancelFail

Error Message:
expected:<200> but was:<404>

Stack Trace:
java.lang.AssertionError: expected:<200> but was:<404>
at 
__randomizedtesting.SeedInfo.seed([298784E81E355B5A:4138B1C2CEAF49B6]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at 
org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.cancelDelegationToken(TestSolrCloudWithDelegationTokens.java:137)
at 
org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.testDelegationTokenCancelFail(TestSolrCloudWithDelegationTokens.java:292)
at 
jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9-ea/Native 
Method)
at 
jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9-ea/NativeMethodAccessorImpl.java:62)
at 
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9-ea/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@9-ea/Method.java:535)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 

[jira] [Commented] (SOLR-9038) Support snapshot management functionality for a solr collection

[Varun Thacker (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15451189#comment-15451189
 ] 

Varun Thacker commented on SOLR-9038:
-

Also feel free to comment on 
https://cwiki.apache.org/confluence/display/solr/Making+and+Restoring+Backups 
where I've tried documenting the feature.

> Support snapshot management functionality for a solr collection
> ---
>
> Key: SOLR-9038
> URL: https://issues.apache.org/jira/browse/SOLR-9038
> Project: Solr
>  Issue Type: New Feature
>  Components: SolrCloud
>Reporter: Hrishikesh Gadre
>Assignee: David Smiley
>
> Currently work is under-way to implement backup/restore API for Solr cloud 
> (SOLR-5750). SOLR-5750 is about providing an ability to "copy" index files 
> and collection metadata to a configurable location. 
> In addition to this, we should also provide a facility to create "named" 
> snapshots for Solr collection. Here by "snapshot" I mean configuring the 
> underlying Lucene IndexDeletionPolicy to not delete a specific commit point 
> (e.g. using PersistentSnapshotIndexDeletionPolicy). This should not be 
> confused with SOLR-5340 which implements core level "backup" functionality.
> The primary motivation of this feature is to decouple recording/preserving a 
> known consistent state of a collection from actually "copying" the relevant 
> files to a physically separate location. This decoupling have number of 
> advantages
> - We can use specialized data-copying tools for transferring Solr index 
> files. e.g. in Hadoop environment, typically 
> [distcp|https://hadoop.apache.org/docs/r1.2.1/distcp2.html] tool is used to 
> copy files from one location to other. This tool provides various options to 
> configure degree of parallelism, bandwidth usage as well as integration with 
> different types and versions of file systems (e.g. AWS S3, Azure Blob store 
> etc.)
> - This separation of concern would also help Solr to focus on the key 
> functionality (i.e. querying and indexing) while delegating the copy 
> operation to the tools built for that purpose.
> - Users can decide if/when to copy the data files as against creating a 
> snapshot. e.g. a user may want to create a snapshot of a collection before 
> making an experimental change (e.g. updating/deleting docs, schema change 
> etc.). If the experiment is successful, he can delete the snapshot (without 
> having to copy the files). If the experiment is failed, then he can copy the 
> files associated with the snapshot and restore.
> Note that Apache Blur project is also providing a similar feature 
> [BLUR-132|https://issues.apache.org/jira/browse/BLUR-132]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9038) Support snapshot management functionality for a solr collection

[Varun Thacker (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15451184#comment-15451184
 ] 

Varun Thacker commented on SOLR-9038:
-

bq. Yes currently we don't support restoring a snapshot directly into the same 
index although it would be a nice feature to add

+1 . I think that will bring out the best of the snapshot API . The user simply 
creates a snapshot , no index files need to be copied . He can then simple 
restore a snapshot and it will be faster than the backup/restore process

> Support snapshot management functionality for a solr collection
> ---
>
> Key: SOLR-9038
> URL: https://issues.apache.org/jira/browse/SOLR-9038
> Project: Solr
>  Issue Type: New Feature
>  Components: SolrCloud
>Reporter: Hrishikesh Gadre
>Assignee: David Smiley
>
> Currently work is under-way to implement backup/restore API for Solr cloud 
> (SOLR-5750). SOLR-5750 is about providing an ability to "copy" index files 
> and collection metadata to a configurable location. 
> In addition to this, we should also provide a facility to create "named" 
> snapshots for Solr collection. Here by "snapshot" I mean configuring the 
> underlying Lucene IndexDeletionPolicy to not delete a specific commit point 
> (e.g. using PersistentSnapshotIndexDeletionPolicy). This should not be 
> confused with SOLR-5340 which implements core level "backup" functionality.
> The primary motivation of this feature is to decouple recording/preserving a 
> known consistent state of a collection from actually "copying" the relevant 
> files to a physically separate location. This decoupling have number of 
> advantages
> - We can use specialized data-copying tools for transferring Solr index 
> files. e.g. in Hadoop environment, typically 
> [distcp|https://hadoop.apache.org/docs/r1.2.1/distcp2.html] tool is used to 
> copy files from one location to other. This tool provides various options to 
> configure degree of parallelism, bandwidth usage as well as integration with 
> different types and versions of file systems (e.g. AWS S3, Azure Blob store 
> etc.)
> - This separation of concern would also help Solr to focus on the key 
> functionality (i.e. querying and indexing) while delegating the copy 
> operation to the tools built for that purpose.
> - Users can decide if/when to copy the data files as against creating a 
> snapshot. e.g. a user may want to create a snapshot of a collection before 
> making an experimental change (e.g. updating/deleting docs, schema change 
> etc.). If the experiment is successful, he can delete the snapshot (without 
> having to copy the files). If the experiment is failed, then he can copy the 
> files associated with the snapshot and restore.
> Note that Apache Blur project is also providing a similar feature 
> [BLUR-132|https://issues.apache.org/jira/browse/BLUR-132]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9038) Support snapshot management functionality for a solr collection

[Hrishikesh Gadre (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15451147#comment-15451147
 ] 

Hrishikesh Gadre commented on SOLR-9038:


[~varunthacker] Yes currently we don't support restoring a snapshot directly 
into the same index although it would be a nice feature to add :)

I need to update the patch for SOLR-9326. Let me do that in a day or so.

> Support snapshot management functionality for a solr collection
> ---
>
> Key: SOLR-9038
> URL: https://issues.apache.org/jira/browse/SOLR-9038
> Project: Solr
>  Issue Type: New Feature
>  Components: SolrCloud
>Reporter: Hrishikesh Gadre
>Assignee: David Smiley
>
> Currently work is under-way to implement backup/restore API for Solr cloud 
> (SOLR-5750). SOLR-5750 is about providing an ability to "copy" index files 
> and collection metadata to a configurable location. 
> In addition to this, we should also provide a facility to create "named" 
> snapshots for Solr collection. Here by "snapshot" I mean configuring the 
> underlying Lucene IndexDeletionPolicy to not delete a specific commit point 
> (e.g. using PersistentSnapshotIndexDeletionPolicy). This should not be 
> confused with SOLR-5340 which implements core level "backup" functionality.
> The primary motivation of this feature is to decouple recording/preserving a 
> known consistent state of a collection from actually "copying" the relevant 
> files to a physically separate location. This decoupling have number of 
> advantages
> - We can use specialized data-copying tools for transferring Solr index 
> files. e.g. in Hadoop environment, typically 
> [distcp|https://hadoop.apache.org/docs/r1.2.1/distcp2.html] tool is used to 
> copy files from one location to other. This tool provides various options to 
> configure degree of parallelism, bandwidth usage as well as integration with 
> different types and versions of file systems (e.g. AWS S3, Azure Blob store 
> etc.)
> - This separation of concern would also help Solr to focus on the key 
> functionality (i.e. querying and indexing) while delegating the copy 
> operation to the tools built for that purpose.
> - Users can decide if/when to copy the data files as against creating a 
> snapshot. e.g. a user may want to create a snapshot of a collection before 
> making an experimental change (e.g. updating/deleting docs, schema change 
> etc.). If the experiment is successful, he can delete the snapshot (without 
> having to copy the files). If the experiment is failed, then he can copy the 
> files associated with the snapshot and restore.
> Note that Apache Blur project is also providing a similar feature 
> [BLUR-132|https://issues.apache.org/jira/browse/BLUR-132]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9444) Fix path usage for cloud backup/restore

[Hrishikesh Gadre (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15451142#comment-15451142
 ] 

Hrishikesh Gadre commented on SOLR-9444:


[~thetaphi] [~varunthacker] Thanks for the reviews! Let me update the patch 
with following changes

- Rename createURI(URI...) method to resolveURI(...)
- Add a test for verifying the whites-paces in the backup location

> Fix path usage for cloud backup/restore
> ---
>
> Key: SOLR-9444
> URL: https://issues.apache.org/jira/browse/SOLR-9444
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Varun Thacker
> Attachments: SOLR-9444.patch
>
>
> As noted by Uwe on 
> https://issues.apache.org/jira/browse/SOLR-9242?focusedCommentId=15438925=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15438925
>  the usage of URI#getPath is wrong. 
> Creating a Jira to track this better. More details to follow



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-Tests-5.5 - Build # 3 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-5.5/3/

All tests passed

Build Log:
[...truncated 10549 lines...]
[javac] Compiling 626 source files to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.5/solr/build/solr-core/classes/test
[javac] 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.5/solr/core/src/test/org/apache/solr/cloud/PeerSyncReplicationTest.java:11:
 error: package java.util.stream does not exist
[javac] import java.util.stream.Collectors;
[javac]^
[javac] Note: Some input files use or override a deprecated API.
[javac] Note: Recompile with -Xlint:deprecation for details.
[javac] Note: Some input files use unchecked or unsafe operations.
[javac] Note: Recompile with -Xlint:unchecked for details.
[javac] 1 error

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.5/build.xml:750: The 
following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.5/build.xml:694: The 
following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.5/build.xml:59: The 
following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.5/solr/build.xml:233: 
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.5/solr/common-build.xml:534:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.5/lucene/common-build.xml:841:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.5/lucene/common-build.xml:855:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.5/lucene/common-build.xml:1989:
 Compile failed; see the compiler error output for details.

Total time: 33 minutes 26 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-6.x-Windows (64bit/jdk1.8.0_102) - Build # 425 - Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Windows/425/
Java: 64bit/jdk1.8.0_102 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC

2 tests failed.
FAILED:  
org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigAliasReplication

Error Message:
expected:<1> but was:<0>

Stack Trace:
java.lang.AssertionError: expected:<1> but was:<0>
at 
__randomizedtesting.SeedInfo.seed([A2A5B4109A416C55:55D65A485CA9C3B3]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at 
org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigAliasReplication(TestReplicationHandler.java:1331)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)


FAILED:  

[jira] [Updated] (LUCENE-7431) Allow negative pre/post values in SpanNotQuery

[Marc Morissette (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marc Morissette updated LUCENE-7431:

Attachment: LUCENE-7431.patch

> Allow negative pre/post values in SpanNotQuery
> --
>
> Key: LUCENE-7431
> URL: https://issues.apache.org/jira/browse/LUCENE-7431
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: core/search
>Reporter: Marc Morissette
>Priority: Minor
> Attachments: LUCENE-7431.patch
>
>
> I need to be able to specify a certain range of allowed overlap between the 
> include and exclude parameters of SpanNotQuery.
> Since this behaviour is the inverse of the behaviour implemented by the pre 
> and post constructor arguments, I suggest that this be implemented with 
> negative pre and post values.
> Patch incoming.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-7431) Allow negative pre/post values in SpanNotQuery

[Marc Morissette (JIRA)]

Marc Morissette created LUCENE-7431:
---

 Summary: Allow negative pre/post values in SpanNotQuery
 Key: LUCENE-7431
 URL: https://issues.apache.org/jira/browse/LUCENE-7431
 Project: Lucene - Core
  Issue Type: Improvement
  Components: core/search
Reporter: Marc Morissette
Priority: Minor


I need to be able to specify a certain range of allowed overlap between the 
include and exclude parameters of SpanNotQuery.

Since this behaviour is the inverse of the behaviour implemented by the pre and 
post constructor arguments, I suggest that this be implemented with negative 
pre and post values.

Patch incoming.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9389) HDFS Transaction logs stay open for writes which leaks Xceivers

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450832#comment-15450832
 ] 

David Smiley commented on SOLR-9389:


bq. 100+ collections with 100 shards each, spread across 8 boxes 

[~TimOwen] that would mean ~1250 shards/box; no?  ~312 per Solr process.   Wow!

> HDFS Transaction logs stay open for writes which leaks Xceivers
> ---
>
> Key: SOLR-9389
> URL: https://issues.apache.org/jira/browse/SOLR-9389
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Hadoop Integration, hdfs
>Affects Versions: 6.1, master (7.0)
>Reporter: Tim Owen
>Assignee: Mark Miller
> Fix For: master (7.0), 6.3
>
> Attachments: SOLR-9389.patch
>
>
> The HdfsTransactionLog implementation keeps a Hadoop FSDataOutputStream open 
> for its whole lifetime, which consumes two threads on the HDFS data node 
> server (dataXceiver and packetresponder) even once the Solr tlog has finished 
> being written to.
> This means for a cluster with many indexes on HDFS, the number of Xceivers 
> can keep growing and eventually hit the limit of 4096 on the data nodes. It's 
> especially likely for indexes that have low write rates, because Solr keeps 
> enough tlogs around to contain 100 documents (up to a limit of 10 tlogs). 
> There's also the issue that attempting to write to a finished tlog would be a 
> major bug, so closing it for writes helps catch that.
> Our cluster during testing had 100+ collections with 100 shards each, spread 
> across 8 boxes (each running 4 solr nodes and 1 hdfs data node) and with 3x 
> replication for the tlog files, this meant we hit the xceiver limit fairly 
> easily and had to use the attached patch to ensure tlogs were closed for 
> writes once finished.
> The patch introduces an extra lifecycle state for the tlog, so it can be 
> closed for writes and free up the HDFS resources, while still being available 
> for reading. I've tried to make it as unobtrusive as I could, but there's 
> probably a better way. I have not changed the behaviour of the local disk 
> tlog implementation, because it only consumes a file descriptor regardless of 
> read or write.
> nb We have decided not to use Solr-on-HDFS now, we're using local disk (for 
> various reasons). So I don't have a HDFS cluster to do further testing on 
> this, I'm just contributing the patch which worked for us.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Windows (32bit/jdk1.8.0_102) - Build # 6089 - Still Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/6089/
Java: 32bit/jdk1.8.0_102 -client -XX:+UseConcMarkSweepGC

3 tests failed.
FAILED:  org.apache.solr.handler.TestReplicationHandler.doTestStressReplication

Error Message:
[index.20160831012617185, index.20160831012617716, index.properties, 
replication.properties, snapshot_metadata] expected:<1> but was:<2>

Stack Trace:
java.lang.AssertionError: [index.20160831012617185, index.20160831012617716, 
index.properties, replication.properties, snapshot_metadata] expected:<1> but 
was:<2>
at 
__randomizedtesting.SeedInfo.seed([262FE662129D4106:FD84E6A417B528B5]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at 
org.apache.solr.handler.TestReplicationHandler.checkForSingleIndex(TestReplicationHandler.java:907)
at 
org.apache.solr.handler.TestReplicationHandler.doTestStressReplication(TestReplicationHandler.java:874)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 

[jira] [Commented] (SOLR-9463) SolrJ: Support Converter and make it easier to extend DocumentObjectBinder

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450807#comment-15450807
 ] 

David Smiley commented on SOLR-9463:


Sounds great; patches welcome :-)

> SolrJ: Support Converter and make it easier to extend DocumentObjectBinder
> --
>
> Key: SOLR-9463
> URL: https://issues.apache.org/jira/browse/SOLR-9463
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: jefferyyuan
>Priority: Minor
>  Labels: extensibility, solrj
>
> In our old project, we use Spring-Solr, it provides some good function such 
> as allow us to define converters to serialize java enum to solr string and 
> vice verse, serialize object as json string and vice verse.
> But it doesn't support latest solr, solr cloud and child documents.
> We would like to use pure solrj, but we do like spring solr's converters 
> function.
> Is it possible that SolrJ can support custom convert in SolrJ?
> Also SolrJ should make it easier to extend DocumentObjectBinder, such as make 
> DocField, infocache, getDocFields etc accessible in sub class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (LUCENE-7430) Geo3d test failure

[Karl Wright (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449022#comment-15449022
 ] 

Karl Wright edited comment on LUCENE-7430 at 8/31/16 2:16 AM:
--

[~mikemccand]: I can't make much sense of the failure here.  The BKD path looks 
like this:

{code}
   [junit4]>   explanation:
   [junit4]> target is in leaf _1(7.0.0):C14521 of full reader 
StandardDirectoryReader(segments:5:nrt _1(7.0.0):C14521)
   [junit4]> full BKD path to target doc:
   [junit4]>   Cell(x=-1.0011188543037526 TO 1.0011188543037524 
y=-1.0011167835214163 TO 1.0011082645037634 z=-0.9977622923536127 TO 
0.9977622923536126); Shape relationship = WITHIN; Quantized point within cell = 
true; Unquantized point within cell = true
   [junit4]>   Cell(x=0.0 TO 1.0011188543037524 y=-1.0011167835214163 
TO 1.0011082645037634 z=-0.9977622923536127 TO 0.9977622923536126); Shape 
relationship = OVERLAPS; Quantized point within cell = true; Unquantized point 
within cell = true
   [junit4]>   Cell(x=0.0 TO 1.0011188543037524 y=-1.0011167835214163 
TO 4.661824259954982E-10 z=-0.9977622923536127 TO 0.9977622923536126); Shape 
relationship = OVERLAPS; Quantized point within cell = true; Unquantized point 
within cell = true
   [junit4]>   Cell(x=0.0 TO 1.0011188543037524 y=-1.0011167835214163 
TO 4.661824259954982E-10 z=0.0 TO 0.9977622923536126); Shape relationship = 
OVERLAPS; Quantized point within cell = true; Unquantized point within cell = 
true
   [junit4]>   Cell(x=0.0 TO 0.5921489289992575 y=-1.0011167835214163 
TO 4.661824259954982E-10 z=0.0 TO 0.9977622923536126); Shape relationship = 
OVERLAPS; Quantized point within cell = true; Unquantized point within cell = 
true
   [junit4]> on cell Cell(x=-1.0011188543037526 TO 1.0011188543037524 
y=-1.0011167835214163 TO 1.0011082645037634 z=-0.9977622923536127 TO 
0.9977622923536126); Shape relationship = WITHIN; Quantized point within cell = 
true; Unquantized point within cell = true, wrapped visitor returned 
CELL_CROSSES_QUERY
   [junit4]> on cell Cell(x=0.0 TO 1.0011188543037524 
y=-1.0011167835214163 TO 1.0011082645037634 z=-0.9977622923536127 TO 
0.9977622923536126); Shape relationship = OVERLAPS; Quantized point within cell 
= true; Unquantized point within cell = true, wrapped visitor returned 
CELL_CROSSES_QUERY
   [junit4]> on cell Cell(x=0.0 TO 1.0011188543037524 
y=-1.0011167835214163 TO 4.661824259954982E-10 z=-0.9977622923536127 TO 
0.9977622923536126); Shape relationship = OVERLAPS; Quantized point within cell 
= true; Unquantized point within cell = true, wrapped visitor returned 
CELL_CROSSES_QUERY
   [junit4]> on cell Cell(x=0.0 TO 1.0011188543037524 
y=-1.0011167835214163 TO 4.661824259954982E-10 z=0.0 TO 0.9977622923536126); 
Shape relationship = OVERLAPS; Quantized point within cell = true; Unquantized 
point within cell = true, wrapped visitor returned CELL_CROSSES_QUERY
   [junit4]> on cell Cell(x=0.0 TO 0.5921489289992575 
y=-1.0011167835214163 TO 4.661824259954982E-10 z=0.0 TO 0.9977622923536126); 
Shape relationship = OVERLAPS; Quantized point within cell = true; Unquantized 
point within cell = true, wrapped visitor returned CELL_CROSSES_QUERY
   [junit4]>   leaf visit docID=4438 x=7.323492821176281E-6 
y=-2.3309121299774915E-10 z=0.9977622921205215
{code}

... which all looks fine and basically seems to say that the point in question 
should wind up as a "hit", and yet it does not.  The quantized point definitely 
is considered within the shape, although it is not quite on the world; it sits 
a bit above the North Pole.  Perhaps it is getting overlooked because of that?  
Maybe the starting bounds for the BKD descent need to take the quantized point 
resolution into account on both ends of each axis?



was (Author: kwri...@metacarta.com):
[~mikemccand]: any ideas welcome...

> Geo3d test failure
> --
>
> Key: LUCENE-7430
> URL: https://issues.apache.org/jira/browse/LUCENE-7430
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Karl Wright
>Assignee: Karl Wright
>
> Here's the error msg:
> {code}
>[junit4] Suite: org.apache.lucene.spatial3d.TestGeo3DPoint
>[junit4] IGNOR/A 0.02s J0 | TestGeo3DPoint.testRandomBig
>[junit4]> Assumption #1: 'nightly' test group is disabled (@Nightly())
>[junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestGeo3DPoint 
> -Dtests.method=testRandomMedium -Dtests.seed=B167D2C466D257A0 
> -Dtests.slow=true -Dtests.locale=pt -Dtests.timezone=Etc/GMT+5 
> -Dtests.asserts=true -Dtests.file.encoding=Cp1252
>[junit4] FAILURE 4.06s J0 | TestGeo3DPoint.testRandomMedium <<<
>[junit4]> Throwable #1: java.lang.AssertionError: FAIL: id=4546 should 
> have matched but did not
>[junit4]

[jira] [Comment Edited] (LUCENE-7430) Geo3d test failure

[Karl Wright (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15448165#comment-15448165
 ] 

Karl Wright edited comment on LUCENE-7430 at 8/31/16 2:11 AM:
--

The north rectangle that is misbehaving is a narrow, very tiny rectangle with 
one point at the north pole and the other two points not far different from 
that.  The point is coming up as being not within the shape bounds but as being 
within the shape.  This is to be expected, however, since the point is not 
exactly on the world but the shape is confined to the world.




was (Author: kwri...@metacarta.com):
The north rectangle that is misbehaving is a narrow, very tiny rectangle with 
one point at the north pole and the other two points not far different from 
that.  The point is coming up as being not within the bounds but as being 
within the shape.  But this applies only to the quantized point; the 
unquantized point is fine.

So in this case it would initially appear that we have a test problem.

We generally accept that there are some effects of quantization; once a point 
is quantized it potentially leaves the exact surface of the world and can 
therefore wind up being outside of a shape's bounds.  That appears to be the 
situation here.  The only question is: what to do about it, without making the 
test useless?

> Geo3d test failure
> --
>
> Key: LUCENE-7430
> URL: https://issues.apache.org/jira/browse/LUCENE-7430
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Karl Wright
>Assignee: Karl Wright
>
> Here's the error msg:
> {code}
>[junit4] Suite: org.apache.lucene.spatial3d.TestGeo3DPoint
>[junit4] IGNOR/A 0.02s J0 | TestGeo3DPoint.testRandomBig
>[junit4]> Assumption #1: 'nightly' test group is disabled (@Nightly())
>[junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestGeo3DPoint 
> -Dtests.method=testRandomMedium -Dtests.seed=B167D2C466D257A0 
> -Dtests.slow=true -Dtests.locale=pt -Dtests.timezone=Etc/GMT+5 
> -Dtests.asserts=true -Dtests.file.encoding=Cp1252
>[junit4] FAILURE 4.06s J0 | TestGeo3DPoint.testRandomMedium <<<
>[junit4]> Throwable #1: java.lang.AssertionError: FAIL: id=4546 should 
> have matched but did not
>[junit4]>   shape=GeoWideNorthRectangle: 
> {planetmodel=PlanetModel.WGS84, 
> bottomlat=1.5707949696510948(89.2224138796), 
> leftlon=3.141592653589793(180.0), 
> rightlon=3.1140200112375265(178.42020396319145)}
>[junit4]>   bounds=XYZBounds: [xmin=-1.3551580601679786E-6 
> xmax=1.3551580601679786E-6 ymin=-1.3551580601679786E-6 
> ymax=1.3551580601679786E-6 zmin=0.9977622910211923 zmax=0.9977622930221051]
>[junit4]>   world bounds=( minX=-1.0011188539924791 
> maxX=1.0011188539924791 minY=-1.0011188539924791 maxY=1.0011188539924791 
> minZ=-0.9977622920221051 maxZ=0.9977622920221051
>[junit4]>   quantized point=[X=7.323492821176281E-6, 
> Y=-2.3309121299774915E-10, Z=0.9977622921205215] within shape? true within 
> bounds? false
>[junit4]>   unquantized point=[lat=1.570788986986606, 
> lon=-1.0056566715747591E-117([X=7.323383942914017E-6, 
> Y=-7.364809920694947E-123, Z=0.9977622919954089])] within shape? false within 
> bounds? false
>[junit4]>   docID=4438 deleted?=false
>[junit4]>   query=PointInGeo3DShapeQuery: field=point: Shape: 
> GeoWideNorthRectangle: {planetmodel=PlanetModel.WGS84, 
> bottomlat=1.5707949696510948(89.2224138796), 
> leftlon=3.141592653589793(180.0), 
> rightlon=3.1140200112375265(178.42020396319145)}
> ...
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 154 - Failure

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/154/

4 tests failed.
FAILED:  org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.test

Error Message:
document count mismatch.  control=756 sum(shards)=755 cloudClient=755

Stack Trace:
java.lang.AssertionError: document count mismatch.  control=756 sum(shards)=755 
cloudClient=755
at 
__randomizedtesting.SeedInfo.seed([22B77F52B5601067:AAE340881B9C7D9F]:0)
at org.junit.Assert.fail(Assert.java:93)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1323)
at 
org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.test(ChaosMonkeyNothingIsSafeTest.java:228)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[jira] [Commented] (SOLR-5725) Efficient facets without counts for enum method

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-5725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450784#comment-15450784
 ] 

David Smiley commented on SOLR-5725:


{{facet.exists=true}} looks good to me... but it should be okay to not specify 
facet.method -- it will be enum to support this feature.  I'm cool with it 
being an error if for some reason you set it to something other than enum.

BTW this feature might theoretically be brought into the other facet types in 
the future.  e.g. {noformat}facet.query={! 
facet.exists=true}field:val{noformat}  or {{facet.range.exists=true}}

> Efficient facets without counts for enum method
> ---
>
> Key: SOLR-5725
> URL: https://issues.apache.org/jira/browse/SOLR-5725
> Project: Solr
>  Issue Type: Improvement
>  Components: search
>Reporter: Alexey Kozhemiakin
>Assignee: Mikhail Khludnev
> Fix For: master (7.0), 6.3
>
> Attachments: SOLR-5725-5x.patch, SOLR-5725-master.patch, 
> SOLR-5725.patch, SOLR-5725.patch, SOLR-5725.patch, SOLR-5725.patch, 
> SOLR-5725.patch
>
>
> Shot version:
> This improves performance for facet.method=enum when it's enough to know that 
> facet count>0, for example when you it's when you dynamically populate 
> filters on search form. New method checks if two bitsets intersect instead of 
> counting intersection size.
> Long version:
> We have a dataset containing hundreds of millions of records, we facet by 
> dozens of fields with many of facet-excludes and have relatively small number 
> of unique values in fields, around thousands.
> Before executing search, users work with "advanced search" form, our  goal is 
> to populate dozens of filters with values which are applicable with other 
> selected values, so basically this is a use case for facets with mincount=1, 
> but without need in actual counts.
> Our performance tests showed that facet.method=enum works much better than 
> fc\fcs, probably due to a specific ratio of "docset"\"unique terms count". 
> For example average execution of query time with method fc=1500ms, fcs=2600ms 
> and with enum=280ms. Profiling indicated the majority time for enum was spent 
> on intersecting docsets.
> Hers's a patch that introduces an extension to facet calculation for 
> method=enum. Basically it uses docSetA.intersects(docSetB) instead of 
> docSetA. intersectionSize (docSetB).
> As a result we were able to reduce our average query time from 280ms to 60ms.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7430) Geo3d test failure

[Karl Wright (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450725#comment-15450725
 ] 

Karl Wright commented on LUCENE-7430:
-

Parameters:

{code}
   [junit4]>   shape=GeoWideNorthRectangle: {planetmodel=PlanetModel.WGS84, 
bottomlat=1.5707949696510948(89.2224138796), 
leftlon=3.141592653589793(180.0), 
rightlon=3.1140200112375265(178.42020396319145)}
   [junit4]>   bounds=XYZBounds: [xmin=-1.3551580601679786E-6 
xmax=1.3551580601679786E-6 ymin=-1.3551580601679786E-6 
ymax=1.3551580601679786E-6 zmin=0.9977622910211923 zmax=0.9977622930221051]
   [junit4]>   world bounds=( minX=-1.0011188539924791 
maxX=1.0011188539924791 minY=-1.0011188539924791 maxY=1.0011188539924791 
minZ=-0.9977622920221051 maxZ=0.9977622920221051
   [junit4]>   quantized point=[X=7.323492821176281E-6, 
Y=-2.3309121299774915E-10, Z=0.9977622921205215] within shape? true within 
bounds? false
   [junit4]>   unquantized point=[lat=1.570788986986606, 
lon=-1.0056566715747591E-117([X=7.323383942914017E-6, 
Y=-7.364809920694947E-123, Z=0.9977622919954089])] within shape? false within 
bounds? false
{code}


> Geo3d test failure
> --
>
> Key: LUCENE-7430
> URL: https://issues.apache.org/jira/browse/LUCENE-7430
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Karl Wright
>Assignee: Karl Wright
>
> Here's the error msg:
> {code}
>[junit4] Suite: org.apache.lucene.spatial3d.TestGeo3DPoint
>[junit4] IGNOR/A 0.02s J0 | TestGeo3DPoint.testRandomBig
>[junit4]> Assumption #1: 'nightly' test group is disabled (@Nightly())
>[junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestGeo3DPoint 
> -Dtests.method=testRandomMedium -Dtests.seed=B167D2C466D257A0 
> -Dtests.slow=true -Dtests.locale=pt -Dtests.timezone=Etc/GMT+5 
> -Dtests.asserts=true -Dtests.file.encoding=Cp1252
>[junit4] FAILURE 4.06s J0 | TestGeo3DPoint.testRandomMedium <<<
>[junit4]> Throwable #1: java.lang.AssertionError: FAIL: id=4546 should 
> have matched but did not
>[junit4]>   shape=GeoWideNorthRectangle: 
> {planetmodel=PlanetModel.WGS84, 
> bottomlat=1.5707949696510948(89.2224138796), 
> leftlon=3.141592653589793(180.0), 
> rightlon=3.1140200112375265(178.42020396319145)}
>[junit4]>   bounds=XYZBounds: [xmin=-1.3551580601679786E-6 
> xmax=1.3551580601679786E-6 ymin=-1.3551580601679786E-6 
> ymax=1.3551580601679786E-6 zmin=0.9977622910211923 zmax=0.9977622930221051]
>[junit4]>   world bounds=( minX=-1.0011188539924791 
> maxX=1.0011188539924791 minY=-1.0011188539924791 maxY=1.0011188539924791 
> minZ=-0.9977622920221051 maxZ=0.9977622920221051
>[junit4]>   quantized point=[X=7.323492821176281E-6, 
> Y=-2.3309121299774915E-10, Z=0.9977622921205215] within shape? true within 
> bounds? false
>[junit4]>   unquantized point=[lat=1.570788986986606, 
> lon=-1.0056566715747591E-117([X=7.323383942914017E-6, 
> Y=-7.364809920694947E-123, Z=0.9977622919954089])] within shape? false within 
> bounds? false
>[junit4]>   docID=4438 deleted?=false
>[junit4]>   query=PointInGeo3DShapeQuery: field=point: Shape: 
> GeoWideNorthRectangle: {planetmodel=PlanetModel.WGS84, 
> bottomlat=1.5707949696510948(89.2224138796), 
> leftlon=3.141592653589793(180.0), 
> rightlon=3.1140200112375265(178.42020396319145)}
> ...
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9463) SolrJ: Support Converter and make it easier to extend DocumentObjectBinder

[jefferyyuan (JIRA)]

jefferyyuan created SOLR-9463:
-

 Summary: SolrJ: Support Converter and make it easier to extend 
DocumentObjectBinder
 Key: SOLR-9463
 URL: https://issues.apache.org/jira/browse/SOLR-9463
 Project: Solr
  Issue Type: Improvement
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: jefferyyuan
Priority: Minor


In our old project, we use Spring-Solr, it provides some good function such as 
allow us to define converters to serialize java enum to solr string and vice 
verse, serialize object as json string and vice verse.

But it doesn't support latest solr, solr cloud and child documents.

We would like to use pure solrj, but we do like spring solr's converters 
function.

Is it possible that SolrJ can support custom convert in SolrJ?

Also SolrJ should make it easier to extend DocumentObjectBinder, such as make 
DocField, infocache, getDocFields etc accessible in sub class.





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-6.x-Linux (64bit/jdk1.8.0_102) - Build # 1631 - Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/1631/
Java: 64bit/jdk1.8.0_102 -XX:+UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.core.TestArbitraryIndexDir.testLoadNewIndexDir

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
at 
__randomizedtesting.SeedInfo.seed([AB6986661ECA0760:42333D5E805397C8]:0)
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:783)
at 
org.apache.solr.core.TestArbitraryIndexDir.testLoadNewIndexDir(TestArbitraryIndexDir.java:107)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=*[count(//doc)=1]
xml response was: 

00


request was:q=id:2=standard=0=20=2.2
at 

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 819 - Still Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/819/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

3 tests failed.
FAILED:  org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test

Error Message:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available 
to handle this request:[http://127.0.0.1:47746/oic/c8n_1x3_lf_shard1_replica2]

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: 
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available 
to handle this request:[http://127.0.0.1:47746/oic/c8n_1x3_lf_shard1_replica2]
at 
__randomizedtesting.SeedInfo.seed([45449AB0D94ECAF:8C007671A3688157]:0)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:769)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1161)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1050)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:992)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:753)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:741)
at 
org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.testRf3WithLeaderFailover(LeaderFailoverAfterPartitionTest.java:178)
at 
org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test(LeaderFailoverAfterPartitionTest.java:57)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 

[jira] [Commented] (SOLR-9319) DELETEREPLICA should be able to accept just count and remove replicas intelligenty

[Nitin Sharma (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450410#comment-15450410
 ] 

Nitin Sharma commented on SOLR-9319:


[~noble.paul] Any updates to the patch needed? 

> DELETEREPLICA should be able to accept  just count and remove replicas 
> intelligenty
> ---
>
> Key: SOLR-9319
> URL: https://issues.apache.org/jira/browse/SOLR-9319
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrCloud
>Reporter: Noble Paul
> Fix For: 6.1
>
> Attachments: DeleteReplicaPatch.jpg, Delete_Replica_count_1.jpg, 
> Delete_Replica_invalid.jpg, Delete_Replica_with_only_1replica.jpg, 
> Delete_replica_count2.jpg, Delte_replica_after.jpg, Delte_replica_before.jpg, 
> Old_deletereplica_api_works.jpg, SOLR-9310.patch, SOLR-9319.patch, 
> SOLR-9319.patch, SOLR-9319.patch, SOLR-9319.patch, Screen Shot 2016-08-26 at 
> 12.59.16 PM.png
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9459) Upgrade dependencies

[JIRA]

 [ 
https://issues.apache.org/jira/browse/SOLR-9459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Høydahl updated SOLR-9459:
--
Attachment: commons-lang3.patch

Agree. Here's an initial patch for upgrading to commons-lang3

> Upgrade dependencies
> 
>
> Key: SOLR-9459
> URL: https://issues.apache.org/jira/browse/SOLR-9459
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Petar Tahchiev
> Attachments: commons-lang3.patch
>
>
> Hello,
> my project has more than 400 dependencies and I'm trying to ban the usage of 
> {{commons-collecrtions}} and {{commons-lang}} in favor of 
> {{org.apache.commons:commons-collection4}} and 
> {{org.apache.commons:commons-lang3}}. Unfortunately out of the 400 
> dependencies *only* solr is still using the old {{collections}} and {{lang}} 
> dependencies which are more than 6 years old.
> Is there a specific reason for that? Can you please update to the latest 
> versions:
> http://repo1.maven.org/maven2/org/apache/commons/commons-lang3/
> http://repo1.maven.org/maven2/org/apache/commons/commons-collections4/
> http://repo1.maven.org/maven2/org/apache/commons/commons-configuration2/
> http://repo1.maven.org/maven2/org/apache/commons/commons-io/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9438) Shard split can lose data

[Shalin Shekhar Mangar (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450041#comment-15450041
 ] 

Shalin Shekhar Mangar commented on SOLR-9438:
-

Beasting this test sometimes fails with nodes not recovering even after working 
around SOLR-9440. I finally found the cause:
{code}
  [beaster]   2> 232316 ERROR 
(coreZkRegister-123-thread-1-processing-n:127.0.0.1:54683_ 
x:collection1_shard1_0_replica0 s:shard1_0 c:collection1 r:core_node7) 
[n:127.0.0.1:54683_ c:collection1 s:shard1_0 r:core_node7 
x:collection1_shard1_0_replica0] o.a.s.c.ZkContainer 
:org.apache.solr.common.SolrException: Error getting leader from zk for shard 
shard1_0
  [beaster]   2>at 
org.apache.solr.cloud.ZkController.getLeader(ZkController.java:994)
  [beaster]   2>at 
org.apache.solr.cloud.ZkController.register(ZkController.java:900)
  [beaster]   2>at 
org.apache.solr.cloud.ZkController.register(ZkController.java:843)
  [beaster]   2>at 
org.apache.solr.core.ZkContainer.lambda$registerInZk$0(ZkContainer.java:181)
  [beaster]   2>at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
  [beaster]   2>at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  [beaster]   2>at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  [beaster]   2>at java.lang.Thread.run(Thread.java:745)
  [beaster]   2> Caused by: org.apache.solr.common.SolrException: There is 
conflicting information about the leader of shard: shard1_0 our state 
says:http://127.0.0.1:54683/collection1_shard1_0_replica1/ but zookeeper 
says:http://127.0.0.1:49547/collection1_shard1_0_replica1/
  [beaster]   2>at 
org.apache.solr.cloud.ZkController.getLeader(ZkController.java:975)
  [beaster]   2>... 7 more
{code}

The problem is that restarting the node (which assigns a new port number) 
sometimes confuses the hell out of SolrCloud and then such nodes keep their old 
port number in cluster state and never recover, can't elect leaders etc. I have 
a suspicion that this behavior is intentional. I'll keep digging.

> Shard split can lose data
> -
>
> Key: SOLR-9438
> URL: https://issues.apache.org/jira/browse/SOLR-9438
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 4.10.4, 5.5.2, 6.1
>Reporter: Shalin Shekhar Mangar
>Assignee: Shalin Shekhar Mangar
>  Labels: difficulty-medium, impact-high
> Fix For: master (7.0), 6.3
>
> Attachments: SOLR-9438.patch, SOLR-9438.patch
>
>
> Solr’s shard split can lose documents if the parent/sub-shard leader is 
> killed (or crashes) between the time that the new sub-shard replica is 
> created and before it recovers. In such a case the slice has already been set 
> to ‘recovery’ state, the sub-shard replica comes up, finds that no other 
> replica is up, waits until the leader vote wait time and then proceeds to 
> become the leader as well as publish itself as active. Once that happens the 
> overseer seeing that all replicas of the sub-shard are now ‘active’, sets the 
> parent slice as ‘inactive’ and the new sub-shard as ‘active’.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9452) JsonRecordReader should not deep copy before handler.handle()

[Shalin Shekhar Mangar (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450031#comment-15450031
 ] 

Shalin Shekhar Mangar commented on SOLR-9452:
-

This patch fails the JsonLoaderTest
{code}
java.lang.AssertionError: 
Expected :null
Actual   :c1
 


at 
__randomizedtesting.SeedInfo.seed([9D742CD9111942BA:383F0AD4B762BD16]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:147)
at 
org.apache.solr.handler.JsonLoaderTest.testJsonDocFormat(JsonLoaderTest.java:381)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
{code}

[~noblepaul] - any ideas?

> JsonRecordReader should not deep copy before handler.handle()
> -
>
> Key: SOLR-9452
> URL: https://issues.apache.org/jira/browse/SOLR-9452
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: update
>Reporter: Shalin Shekhar Mangar
> Fix For: master (7.0), 6.3
>
> Attachments: SOLR-9452.patch
>
>
> JsonRecordReader does a deep copy of the document map before calling 
> handler.handle() method but it is not required because it is consumed in the 
> same thread and not stored anywhere. The only place which needs a deep copy 
> is the JsonRecordReader#getAllRecords method (used mostly for testing). Any 
> such method can perform deep copy itself so that the common case is not 
> penalized.
> This will save allocation of one copy of the map for each document.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+132) - Build # 17726 - Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/17726/
Java: 32bit/jdk-9-ea+132 -client -XX:+UseG1GC

1 tests failed.
FAILED:  org.apache.solr.security.BasicAuthIntegrationTest.testBasics

Error Message:
IOException occured when talking to server at: http://127.0.0.1:33755/solr

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: IOException occured when 
talking to server at: http://127.0.0.1:33755/solr
at 
__randomizedtesting.SeedInfo.seed([A5DC99AC24698849:980437801C87D639]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:622)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:261)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:250)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:415)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:367)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1280)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1050)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:992)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at 
org.apache.solr.security.BasicAuthIntegrationTest.doExtraTests(BasicAuthIntegrationTest.java:116)
at 
org.apache.solr.cloud.TestMiniSolrCloudClusterBase.testCollectionCreateSearchDelete(TestMiniSolrCloudClusterBase.java:196)
at 
org.apache.solr.cloud.TestMiniSolrCloudClusterBase.testBasics(TestMiniSolrCloudClusterBase.java:79)
at 
jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9-ea/Native 
Method)
at 
jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9-ea/NativeMethodAccessorImpl.java:62)
at 
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9-ea/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@9-ea/Method.java:535)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 

[jira] [Resolved] (SOLR-9447) Do not clone SolrInputDocument if update processor chain does not contain custom processors

[Shalin Shekhar Mangar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shalin Shekhar Mangar resolved SOLR-9447.
-
Resolution: Fixed

> Do not clone SolrInputDocument if update processor chain does not contain 
> custom processors
> ---
>
> Key: SOLR-9447
> URL: https://issues.apache.org/jira/browse/SOLR-9447
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: Shalin Shekhar Mangar
>Assignee: Shalin Shekhar Mangar
> Fix For: master (7.0), 6.3
>
> Attachments: SOLR-9447.patch
>
>
> In SOLR-3215 we started cloning documents before adding them locally and send 
> the cloned copy to replicas. This was done because processors after 
> DistributedUpdateProcessor can affect the docs before they are sent to the 
> replicas.
> However, we can avoid the deep copy if we know for sure that the processors 
> after DUP are one of (LogUpdateProcessor, RunUpdateProcessor, 
> TolerantUpdateProcessor) which definitely do not modify the document. This 
> ensures that the common case is optimized.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9447) Do not clone SolrInputDocument if update processor chain does not contain custom processors

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449955#comment-15449955
 ] 

ASF subversion and git services commented on SOLR-9447:
---

Commit 65c2cf3a0a260d571d0da7c50fb8295f1473b9a3 in lucene-solr's branch 
refs/heads/branch_6x from [~shalinmangar]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=65c2cf3 ]

SOLR-9447: Do not clone SolrInputDocument if update processor chain does not 
contain custom processors.
(cherry picked from commit 26262f4)


> Do not clone SolrInputDocument if update processor chain does not contain 
> custom processors
> ---
>
> Key: SOLR-9447
> URL: https://issues.apache.org/jira/browse/SOLR-9447
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: Shalin Shekhar Mangar
>Assignee: Shalin Shekhar Mangar
> Fix For: master (7.0), 6.3
>
> Attachments: SOLR-9447.patch
>
>
> In SOLR-3215 we started cloning documents before adding them locally and send 
> the cloned copy to replicas. This was done because processors after 
> DistributedUpdateProcessor can affect the docs before they are sent to the 
> replicas.
> However, we can avoid the deep copy if we know for sure that the processors 
> after DUP are one of (LogUpdateProcessor, RunUpdateProcessor, 
> TolerantUpdateProcessor) which definitely do not modify the document. This 
> ensures that the common case is optimized.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9447) Do not clone SolrInputDocument if update processor chain does not contain custom processors

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449953#comment-15449953
 ] 

ASF subversion and git services commented on SOLR-9447:
---

Commit 26262f4074d43c3167efcfb83452daf081b0d9b9 in lucene-solr's branch 
refs/heads/master from [~shalinmangar]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=26262f4 ]

SOLR-9447: Do not clone SolrInputDocument if update processor chain does not 
contain custom processors.


> Do not clone SolrInputDocument if update processor chain does not contain 
> custom processors
> ---
>
> Key: SOLR-9447
> URL: https://issues.apache.org/jira/browse/SOLR-9447
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: Shalin Shekhar Mangar
>Assignee: Shalin Shekhar Mangar
> Fix For: master (7.0), 6.3
>
> Attachments: SOLR-9447.patch
>
>
> In SOLR-3215 we started cloning documents before adding them locally and send 
> the cloned copy to replicas. This was done because processors after 
> DistributedUpdateProcessor can affect the docs before they are sent to the 
> replicas.
> However, we can avoid the deep copy if we know for sure that the processors 
> after DUP are one of (LogUpdateProcessor, RunUpdateProcessor, 
> TolerantUpdateProcessor) which definitely do not modify the document. This 
> ensures that the common case is optimized.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5725) Efficient facets without counts for enum method

[Mikhail Khludnev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-5725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Khludnev updated SOLR-5725:
---
Attachment: SOLR-5725.patch

bq. What's with the rb.req.getParams() at the top of 
modifyRequestForFieldFacets?
ops.. I'm sorry. I started type there, but then all magic went to 
{{.FacetComponent.EnumProbingDistribFacetField}}. Fixed in attached patch.
bq. for the loop label in SimpleFacets.getFacetTermEnumCounts, can you please 
use all-caps 
Sure. Fixed in patch. 
bq. In TestRandomFaceting.getExpectationForSortByCount: why are offset & limit 
multiplied by 2?
facets in json response come as a flat list like \["a", 1, "c", 1, "e", 4], 
thus every tuple takes a pair of elems in array.

Ok. Let's confirm the spec:
{panel}
{{facet.exists=true}} is valid only for {{facet.method=enum}}, i.e. it throws 
exceptions on other methods. It caps facet counts by 1, hopefully does it 
faster than usual {{enum}}.
{panel}  
Does it work for everyone?  

> Efficient facets without counts for enum method
> ---
>
> Key: SOLR-5725
> URL: https://issues.apache.org/jira/browse/SOLR-5725
> Project: Solr
>  Issue Type: Improvement
>  Components: search
>Reporter: Alexey Kozhemiakin
>Assignee: Mikhail Khludnev
> Fix For: master (7.0), 6.3
>
> Attachments: SOLR-5725-5x.patch, SOLR-5725-master.patch, 
> SOLR-5725.patch, SOLR-5725.patch, SOLR-5725.patch, SOLR-5725.patch, 
> SOLR-5725.patch
>
>
> Shot version:
> This improves performance for facet.method=enum when it's enough to know that 
> facet count>0, for example when you it's when you dynamically populate 
> filters on search form. New method checks if two bitsets intersect instead of 
> counting intersection size.
> Long version:
> We have a dataset containing hundreds of millions of records, we facet by 
> dozens of fields with many of facet-excludes and have relatively small number 
> of unique values in fields, around thousands.
> Before executing search, users work with "advanced search" form, our  goal is 
> to populate dozens of filters with values which are applicable with other 
> selected values, so basically this is a use case for facets with mincount=1, 
> but without need in actual counts.
> Our performance tests showed that facet.method=enum works much better than 
> fc\fcs, probably due to a specific ratio of "docset"\"unique terms count". 
> For example average execution of query time with method fc=1500ms, fcs=2600ms 
> and with enum=280ms. Profiling indicated the majority time for enum was spent 
> on intersecting docsets.
> Hers's a patch that introduces an extension to facet calculation for 
> method=enum. Basically it uses docSetA.intersects(docSetB) instead of 
> docSetA. intersectionSize (docSetB).
> As a result we were able to reduce our average query time from 280ms to 60ms.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Status of Solr Ref Guide for 6.2

[Cassandra Targett]

OK, great then.

Another issue, however, is the PDF seems to be missing the logo on the
title page. This wasn't a problem for the last release (I checked it
again to be sure).

I checked the location of the logo image file, and it is still
correct. I also checked the intermediate HTML and it contains the
correct reference to the file location. However, the HTML refers to
the online image, so when I view that I see the logo because it's
accessible to me. Somehow it's not getting downloaded to be included
with the PDF when that is generated.

The PDF seemed to take a very long time to generate (I did it 2x), so
it's possible there is some network issue causing a problem. I
recommend waiting a few hours, or until tomorrow morning, and checking
again. If it's still a problem, we should raise an issue with INFRA
for investigation.

On Tue, Aug 30, 2016 at 8:17 AM, Varun Thacker  wrote:
> Hi Cassandra,
>
> Uwe's already made the changes .
>
>
> Regarding the RC:
> I had a doubt when I was documenting SOLR-9038 so I'll wait to hear back on
> the Jira. Hopefully I can wrap that up soon and then cut an RC today
>
> On Tue, Aug 30, 2016 at 6:44 PM, Cassandra Targett 
> wrote:
>>
>> Hey Varun,
>>
>> You may want to email Uwe directly without cc'ing the list. He's said
>> in the past that he missed the requests because of his email
>> filtering, and it's better to email him directly so it doesn't get
>> lost.
>>
>> On Tue, Aug 30, 2016 at 3:45 AM, Varun Thacker  wrote:
>> > Hi Uwe,
>> >
>> > Could you please update the CWIKI Javadoc macro links to point to the
>> > 6_2_0
>> > paths.
>> >
>> > I'll start the release process for the ref guide soon afterwards.
>> >
>> > On Tue, Aug 30, 2016 at 2:06 AM, Joel Bernstein 
>> > wrote:
>> >>
>> >> Hi Varun,
>> >>
>> >> I didn't get a chance yet to document the new streaming expressions.
>> >> I'm
>> >> on vacation this week. I'm planning on updating the docs early next
>> >> week. If
>> >> the docs release before then, people can read about the new streaming
>> >> expressions online.
>> >>
>> >>
>> >> On Aug 29, 2016 10:13 AM, "Varun Thacker"  wrote:
>> >>>
>> >>> Hi Everyone,
>> >>>
>> >>> I think the majority of the changes are in place. I will try to make
>> >>> the
>> >>> necessary changes required for SOLR-9187/SOLR-9038/SOLR-9243 later
>> >>> today.
>> >>>
>> >>> I'm not sure where in
>> >>> https://cwiki.apache.org/confluence/display/solr/Using+SolrJ can we
>> >>> document
>> >>> SOLR-9090  . It seems like we need to beef up the SolrJ page in
>> >>> general?
>> >>>
>> >>> Joel, have you been able to add the new features in streaming
>> >>> expressions
>> >>> to the ref guide? I can help review it
>> >>>
>> >>> I will aim to create an RC tomorrow morning in IST hours.
>> >>>
>> >>> On Thu, Aug 25, 2016 at 12:30 PM, Varun Thacker 
>> >>> wrote:
>> 
>>  Hi Cassandra,
>> 
>>  I can volunteer to be the RM for the ref guide.
>> 
>>  We probably won't get to all the TODOs but I think let's start
>>  working
>>  on it for the next few days.
>> 
>>  If its fine we can cut an RC on Monday 29th August and then have the
>>  ref
>>  guide released later in the week.
>> 
>>  On Thu, Aug 25, 2016 at 11:23 AM, Noble Paul 
>>  wrote:
>> >
>> > I shall document my changes today itself
>> >
>> > On Thu, Aug 25, 2016 at 3:39 AM, Joel Bernstein 
>> > wrote:
>> > > Hi Cassandra,
>> > >
>> > > I'm also behind on documentation for this release and on vacation
>> > > next week.
>> > > But I will attempt to make progress on the docs this week and do
>> > > some
>> > > documentation while on vacation as well.
>> > >
>> > > Joel Bernstein
>> > > http://joelsolr.blogspot.com/
>> > >
>> > > On Wed, Aug 24, 2016 at 5:49 PM, Cassandra Targett
>> > > 
>> > > wrote:
>> > >>
>> > >> The vote for Lucene/Solr 6.2 has passed already, but the release
>> > >> process for the Ref Guide has barely started.
>> > >>
>> > >> I'm in an offsite meeting this week, and next week have another
>> > >> project to focus on. Hoss is offline until early September.
>> > >> So...the
>> > >> 2
>> > >> people who normally do the majority of the work to prepare and
>> > >> publish
>> > >> the Ref Guide aren't available.
>> > >>
>> > >> I've made a rather barebones TODO list, pulled only from the New
>> > >> Features section of CHANGES (other sections should be reviewed
>> > >> also,
>> > >> however. I won't really have time to do much more, and can't be
>> > >> the
>> > >> Release Manager right now.
>> > >>
>> > >> Essentially, if you care to have a 6.2 Ref Guide, I'm asking for
>> > >> volunteers for any/all of the following:
>> 

Re: Lucene or Apache Solr : Project Decision making

[Doug Turnbull]

Hi Archit, I would make a strong argument for using Solr unless you have
some exotic requirements.

- Solr has distributed indexing and search built in, building your own
distributed system is non-trivial, just as Mark Miller :)
- Solr comes prebaked with an HTTP API for non search experts to interact
with.
- For hiring, it's more likely you'll find a Solr expert than a Lucene
expert
- Custom capabilities can be handled by Solr plugins that specialize bits
and pieces of Solr to your needs
- You can pretty easily proxy Solr for security, from anything from a dumb
nginx proxy to a tad bit of custom code

I might consider using just Lucene if the consumers of my library don't
realize there's "search" under the hood
- I really just want a Java library that does search-like operations under
the hood, but the consumers of my code don't care about search.
- I'm doing something data-sciency with Lucene, my problem doesn't resemble
search, and I want direct control (ie classification, etc).

(note Elasticsearch would have similar capabilities and pros/cons vs Solr,
but the Solr vs ES is a whole 'nother conversation and I don't want to
hijack your thread)

-Doug



On Tue, Aug 30, 2016 at 1:24 PM Alexandre Rafalovitch 
wrote:

> SolerCloud uses Zookeeper. As to the rest, Solr is open source. It may be
> more efficient stripping out whatever you don't want than reinventing it on
> top of Lucene again.
>
> Regards,
> Alex
>
> On 31 Aug 2016 12:17 AM, "archit mehta"  wrote:
>
>> Hi,
>>
>> We need to take decision whether to go for lucene or solr. There are few
>> points which I would like to mention.
>>
>> 1. If we use lucene we do not have to worry about security as it is
>> already taken care but need to build own distributed indexer and searcher,
>> if we use solr then we don't have to worry about distributed indexer and
>> searcher but as it is a another process we have to put some security
>> controls.
>>
>> In our case getting permission for solr is bit difficult, lucene is
>> already in production (withou distribution stuff)
>>
>> 2. Does solr uses kafka or zookeeper or other third party library? Can I
>> get list from somewhere?
>> Server is heavily loaded, new process and running kafka/zookeeper is also
>> an overhead for us.
>> With current implementation we removed kafka and wrote some of our own
>> code.
>>
>> How much easy or difficult to build distributed indexer and searcher with
>> core lucene?
>>
>> Kindly share your views based on the point I have mentioned here. In case
>> any more clarification require write me back.
>>
>>
>> Regards,
>> Archit
>>
>>

[jira] [Resolved] (SOLR-9439) Shard split clean up logic for older failed splits is faulty

[Shalin Shekhar Mangar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shalin Shekhar Mangar resolved SOLR-9439.
-
Resolution: Fixed

Thanks Steve!

> Shard split clean up logic for older failed splits is faulty
> 
>
> Key: SOLR-9439
> URL: https://issues.apache.org/jira/browse/SOLR-9439
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 4.10.4, 5.5.2, 6.1
>Reporter: Shalin Shekhar Mangar
>Assignee: Shalin Shekhar Mangar
> Fix For: master (7.0), 6.3
>
> Attachments: Lucene-Solr-tests-master.8015.log.gz, 
> SOLR-9439-fix-deleteshard.patch, SOLR-9439-fix-deleteshard.patch, 
> SOLR-9439.patch, SOLR-9439.patch, SOLR-9439.patch
>
>
> In case a split finds that previous sub-shards exist in construction or 
> recovery state. it tries to clean them up by invoking deleteshard API. 
> However, the clean up logic tries to invoke deleteshard on the same 
> sub-shards as many times as the requested number of sub-ranges. Such repeat 
> calls to deleteshard fail and therefore fail the entire shard split operation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9439) Shard split clean up logic for older failed splits is faulty

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449719#comment-15449719
 ] 

ASF subversion and git services commented on SOLR-9439:
---

Commit 6bf9513b9385a53557dc0849eb36a062aceb8e8c in lucene-solr's branch 
refs/heads/branch_6x from [~shalinmangar]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=6bf9513 ]

SOLR-9439: The delete shard API has been made more resilient against failures 
resulting from non-existent cores.
(cherry picked from commit 02b97a2)


> Shard split clean up logic for older failed splits is faulty
> 
>
> Key: SOLR-9439
> URL: https://issues.apache.org/jira/browse/SOLR-9439
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 4.10.4, 5.5.2, 6.1
>Reporter: Shalin Shekhar Mangar
>Assignee: Shalin Shekhar Mangar
> Fix For: master (7.0), 6.3
>
> Attachments: Lucene-Solr-tests-master.8015.log.gz, 
> SOLR-9439-fix-deleteshard.patch, SOLR-9439-fix-deleteshard.patch, 
> SOLR-9439.patch, SOLR-9439.patch, SOLR-9439.patch
>
>
> In case a split finds that previous sub-shards exist in construction or 
> recovery state. it tries to clean them up by invoking deleteshard API. 
> However, the clean up logic tries to invoke deleteshard on the same 
> sub-shards as many times as the requested number of sub-ranges. Such repeat 
> calls to deleteshard fail and therefore fail the entire shard split operation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9439) Shard split clean up logic for older failed splits is faulty

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449717#comment-15449717
 ] 

ASF subversion and git services commented on SOLR-9439:
---

Commit 02b97a29b747e439bba8ad95a0269f959bea965e in lucene-solr's branch 
refs/heads/master from [~shalinmangar]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=02b97a2 ]

SOLR-9439: The delete shard API has been made more resilient against failures 
resulting from non-existent cores.


> Shard split clean up logic for older failed splits is faulty
> 
>
> Key: SOLR-9439
> URL: https://issues.apache.org/jira/browse/SOLR-9439
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 4.10.4, 5.5.2, 6.1
>Reporter: Shalin Shekhar Mangar
>Assignee: Shalin Shekhar Mangar
> Fix For: master (7.0), 6.3
>
> Attachments: Lucene-Solr-tests-master.8015.log.gz, 
> SOLR-9439-fix-deleteshard.patch, SOLR-9439-fix-deleteshard.patch, 
> SOLR-9439.patch, SOLR-9439.patch, SOLR-9439.patch
>
>
> In case a split finds that previous sub-shards exist in construction or 
> recovery state. it tries to clean them up by invoking deleteshard API. 
> However, the clean up logic tries to invoke deleteshard on the same 
> sub-shards as many times as the requested number of sub-ranges. Such repeat 
> calls to deleteshard fail and therefore fail the entire shard split operation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9439) Shard split clean up logic for older failed splits is faulty

[Shalin Shekhar Mangar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shalin Shekhar Mangar updated SOLR-9439:

Attachment: SOLR-9439-fix-deleteshard.patch

The last patch failed CollectionsAPISolrJTest.testCreateAndDeleteShard because 
the changed implementation did not return the "success" flag. I beasted this 
test 50 times but couldn't get it to fail.

> Shard split clean up logic for older failed splits is faulty
> 
>
> Key: SOLR-9439
> URL: https://issues.apache.org/jira/browse/SOLR-9439
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 4.10.4, 5.5.2, 6.1
>Reporter: Shalin Shekhar Mangar
>Assignee: Shalin Shekhar Mangar
> Fix For: master (7.0), 6.3
>
> Attachments: Lucene-Solr-tests-master.8015.log.gz, 
> SOLR-9439-fix-deleteshard.patch, SOLR-9439-fix-deleteshard.patch, 
> SOLR-9439.patch, SOLR-9439.patch, SOLR-9439.patch
>
>
> In case a split finds that previous sub-shards exist in construction or 
> recovery state. it tries to clean them up by invoking deleteshard API. 
> However, the clean up logic tries to invoke deleteshard on the same 
> sub-shards as many times as the requested number of sub-ranges. Such repeat 
> calls to deleteshard fail and therefore fail the entire shard split operation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Lucene or Apache Solr : Project Decision making

[Alexandre Rafalovitch]

SolerCloud uses Zookeeper. As to the rest, Solr is open source. It may be
more efficient stripping out whatever you don't want than reinventing it on
top of Lucene again.

Regards,
Alex

On 31 Aug 2016 12:17 AM, "archit mehta"  wrote:

> Hi,
>
> We need to take decision whether to go for lucene or solr. There are few
> points which I would like to mention.
>
> 1. If we use lucene we do not have to worry about security as it is
> already taken care but need to build own distributed indexer and searcher,
> if we use solr then we don't have to worry about distributed indexer and
> searcher but as it is a another process we have to put some security
> controls.
>
> In our case getting permission for solr is bit difficult, lucene is
> already in production (withou distribution stuff)
>
> 2. Does solr uses kafka or zookeeper or other third party library? Can I
> get list from somewhere?
> Server is heavily loaded, new process and running kafka/zookeeper is also
> an overhead for us.
> With current implementation we removed kafka and wrote some of our own
> code.
>
> How much easy or difficult to build distributed indexer and searcher with
> core lucene?
>
> Kindly share your views based on the point I have mentioned here. In case
> any more clarification require write me back.
>
>
> Regards,
> Archit
>
>

Lucene or Apache Solr : Project Decision making

[archit mehta]

Hi,

We need to take decision whether to go for lucene or solr. There are few
points which I would like to mention.

1. If we use lucene we do not have to worry about security as it is already
taken care but need to build own distributed indexer and searcher, if we
use solr then we don't have to worry about distributed indexer and searcher
but as it is a another process we have to put some security controls.

In our case getting permission for solr is bit difficult, lucene is already
in production (withou distribution stuff)

2. Does solr uses kafka or zookeeper or other third party library? Can I
get list from somewhere?
Server is heavily loaded, new process and running kafka/zookeeper is also
an overhead for us.
With current implementation we removed kafka and wrote some of our own code.

How much easy or difficult to build distributed indexer and searcher with
core lucene?

Kindly share your views based on the point I have mentioned here. In case
any more clarification require write me back.


Regards,
Archit

[jira] [Created] (SOLR-9462) Add cache metrics for collection alias

[Michael Sun (JIRA)]

Michael Sun created SOLR-9462:
-

 Summary: Add cache metrics for collection alias
 Key: SOLR-9462
 URL: https://issues.apache.org/jira/browse/SOLR-9462
 Project: Solr
  Issue Type: Improvement
  Security Level: Public (Default Security Level. Issues are Public)
  Components: JMX
Affects Versions: 6.2
Reporter: Michael Sun


Solr caches are major memory consumers and Solr cache metrics are critical in 
memory tuning. However in current implementation, only cache metrics for 
individual collection is available. To get the same metrics for a collection 
alias, which may cover tens collections, the only way is to aggregate metrics 
from each collection for the collection alias manually or using an external 
tool.

The JIRA is to provide cache metrics for a collection alias to simplify the 
memory tuning for collection alias.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Windows (64bit/jdk1.8.0_102) - Build # 6088 - Still Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/6088/
Java: 64bit/jdk1.8.0_102 -XX:-UseCompressedOops -XX:+UseG1GC

5 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.CdcrVersionReplicationTest

Error Message:
ObjectTracker found 1 object(s) that were not released!!! [InternalHttpClient]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 1 object(s) that were not 
released!!! [InternalHttpClient]
at __randomizedtesting.SeedInfo.seed([8C2735F134358164]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at 
org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:257)
at sun.reflect.GeneratedMethodAccessor21.invoke(Unknown Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)


FAILED:  
org.apache.solr.common.cloud.TestCollectionStateWatchers.testCanWaitForNonexistantCollection

Error Message:
waitForState was not triggered by collection creation

Stack Trace:
java.lang.AssertionError: waitForState was not triggered by collection creation
at 
__randomizedtesting.SeedInfo.seed([F419BC5088E1FAC9:5F3927E6E5BF3EE2]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.common.cloud.TestCollectionStateWatchers.testCanWaitForNonexistantCollection(TestCollectionStateWatchers.java:207)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 

[jira] [Commented] (SOLR-5725) Efficient facets without counts for enum method

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-5725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449307#comment-15449307
 ] 

David Smiley commented on SOLR-5725:


bq. facet.maxcount=1

I don't like that suggestion for two reasons:  (1) the maxcount name might 
suggest we don't return values that would have a higher count -- and that's not 
true, and (2) This feature is for using just '1'... and this parameter suggests 
you can set it to some other number and we don't support that.

> Efficient facets without counts for enum method
> ---
>
> Key: SOLR-5725
> URL: https://issues.apache.org/jira/browse/SOLR-5725
> Project: Solr
>  Issue Type: Improvement
>  Components: search
>Reporter: Alexey Kozhemiakin
>Assignee: Mikhail Khludnev
> Fix For: master (7.0), 6.3
>
> Attachments: SOLR-5725-5x.patch, SOLR-5725-master.patch, 
> SOLR-5725.patch, SOLR-5725.patch, SOLR-5725.patch, SOLR-5725.patch
>
>
> Shot version:
> This improves performance for facet.method=enum when it's enough to know that 
> facet count>0, for example when you it's when you dynamically populate 
> filters on search form. New method checks if two bitsets intersect instead of 
> counting intersection size.
> Long version:
> We have a dataset containing hundreds of millions of records, we facet by 
> dozens of fields with many of facet-excludes and have relatively small number 
> of unique values in fields, around thousands.
> Before executing search, users work with "advanced search" form, our  goal is 
> to populate dozens of filters with values which are applicable with other 
> selected values, so basically this is a use case for facets with mincount=1, 
> but without need in actual counts.
> Our performance tests showed that facet.method=enum works much better than 
> fc\fcs, probably due to a specific ratio of "docset"\"unique terms count". 
> For example average execution of query time with method fc=1500ms, fcs=2600ms 
> and with enum=280ms. Profiling indicated the majority time for enum was spent 
> on intersecting docsets.
> Hers's a patch that introduces an extension to facet calculation for 
> method=enum. Basically it uses docSetA.intersects(docSetB) instead of 
> docSetA. intersectionSize (docSetB).
> As a result we were able to reduce our average query time from 280ms to 60ms.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7407) Explore switching doc values to an iterator API

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449308#comment-15449308
 ] 

Michael McCandless commented on LUCENE-7407:


Higher level as well: rather than a random access {{NumericDocValues}}, you 
have a {{NumericDocValuesIterator}} extending {{DocIdSetIterator}} and just 
adding a {{longValue}} method, for example.

> Explore switching doc values to an iterator API
> ---
>
> Key: LUCENE-7407
> URL: https://issues.apache.org/jira/browse/LUCENE-7407
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Michael McCandless
>Assignee: Michael McCandless
>  Labels: docValues
>
> I think it could be compelling if we restricted doc values to use an
> iterator API at read time, instead of the more general random access
> API we have today:
>   * It would make doc values disk usage more of a "you pay for what
> what you actually use", like postings, which is a compelling
> reduction for sparse usage.
>   * I think codecs could compress better and maybe speed up decoding
> of doc values, even in the non-sparse case, since the read-time
> API is more restrictive "forward only" instead of random access.
>   * We could remove {{getDocsWithField}} entirely, since that's
> implicit in the iteration, and the awkward "return 0 if the
> document didn't have this field" would go away.
>   * We can remove the annoying thread locals we must make today in
> {{CodecReader}}, and close the trappy "I accidentally shared a
> single XXXDocValues instance across threads", since an iterator is
> inherently "use once".
>   * We could maybe leverage the numerous optimizations we've done for
> postings over time, since the two problems ("iterate over doc ids
> and store something interesting for each") are very similar.
> This idea has come up many in the past, e.g. LUCENE-7253 is a recent
> example, and very early iterations of doc values started with exactly
> this ;)
> However, it's a truly enormous change, likely 7.0 only.  Or maybe we
> could have the new iterator APIs also ported to 6.x side by side with
> the deprecate existing random-access APIs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5725) Efficient facets without counts for enum method

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-5725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449298#comment-15449298
 ] 

David Smiley commented on SOLR-5725:


Refining what I said before... we don't even _need_ a new facet.method for 
this, although it's fine to have one.

code review:
* What's with the rb.req.getParams() at the top of modifyRequestForFieldFacets?
* for the loop label in SimpleFacets.getFacetTermEnumCounts, can you please use 
all-caps -- {{SEGMENTS}} or {{SEGMENTS_LOOP}}.
* In TestRandomFaceting.getExpectationForSortByCount: why are offset & limit 
multiplied by 2?

(not introduced by this patch):
* Hmmm, I'm looking at the algorithm in SimpleFacets.getFacetTermEnumCounts for 
when the df is < minDfFilterCache -- which loops over docs in the PostingsEnum 
and checks fastForRandomSet.  Wouldn't it be better to leap-frog (and then 
don't need a "fast-for-random-set" but do need it to be sorted)?  What do you 
think [~yo...@apache.org]?

> Efficient facets without counts for enum method
> ---
>
> Key: SOLR-5725
> URL: https://issues.apache.org/jira/browse/SOLR-5725
> Project: Solr
>  Issue Type: Improvement
>  Components: search
>Reporter: Alexey Kozhemiakin
>Assignee: Mikhail Khludnev
> Fix For: master (7.0), 6.3
>
> Attachments: SOLR-5725-5x.patch, SOLR-5725-master.patch, 
> SOLR-5725.patch, SOLR-5725.patch, SOLR-5725.patch, SOLR-5725.patch
>
>
> Shot version:
> This improves performance for facet.method=enum when it's enough to know that 
> facet count>0, for example when you it's when you dynamically populate 
> filters on search form. New method checks if two bitsets intersect instead of 
> counting intersection size.
> Long version:
> We have a dataset containing hundreds of millions of records, we facet by 
> dozens of fields with many of facet-excludes and have relatively small number 
> of unique values in fields, around thousands.
> Before executing search, users work with "advanced search" form, our  goal is 
> to populate dozens of filters with values which are applicable with other 
> selected values, so basically this is a use case for facets with mincount=1, 
> but without need in actual counts.
> Our performance tests showed that facet.method=enum works much better than 
> fc\fcs, probably due to a specific ratio of "docset"\"unique terms count". 
> For example average execution of query time with method fc=1500ms, fcs=2600ms 
> and with enum=280ms. Profiling indicated the majority time for enum was spent 
> on intersecting docsets.
> Hers's a patch that introduces an extension to facet calculation for 
> method=enum. Basically it uses docSetA.intersects(docSetB) instead of 
> docSetA. intersectionSize (docSetB).
> As a result we were able to reduce our average query time from 280ms to 60ms.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

ApacheCon Seville CFP closes September 9th

[Rich Bowen]

It's traditional. We wait for the last minute to get our talk proposals
in for conferences.

Well, the last minute has arrived. The CFP for ApacheCon Seville closes
on September 9th, which is less than 2 weeks away. It's time to get your
talks in, so that we can make this the best ApacheCon yet.

It's also time to discuss with your developer and user community whether
there's a track of talks that you might want to propose, so that you
have more complete coverage of your project than a talk or two.

For Apache Big Data, the relevant URLs are:
Event details:
http://events.linuxfoundation.org/events/apache-big-data-europe
CFP:
http://events.linuxfoundation.org/events/apache-big-data-europe/program/cfp

For ApacheCon Europe, the relevant URLs are:
Event details: http://events.linuxfoundation.org/events/apachecon-europe
CFP: http://events.linuxfoundation.org/events/apachecon-europe/program/cfp

This year, we'll be reviewing papers "blind" - that is, looking at the
abstracts without knowing who the speaker is. This has been shown to
eliminate the "me and my buddies" nature of many tech conferences,
producing more diversity, and more new speakers. So make sure your
abstracts clearly explain what you'll be talking about.

For further updated about ApacheCon, follow us on Twitter, @ApacheCon,
or drop by our IRC channel, #apachecon on the Freenode IRC network.

-- 
Rich Bowen
WWW: http://apachecon.com/
Twitter: @ApacheCon

ApacheCon Seville CFP closes September 9th

[Rich Bowen]

It's traditional. We wait for the last minute to get our talk proposals
in for conferences.

Well, the last minute has arrived. The CFP for ApacheCon Seville closes
on September 9th, which is less than 2 weeks away. It's time to get your
talks in, so that we can make this the best ApacheCon yet.

It's also time to discuss with your developer and user community whether
there's a track of talks that you might want to propose, so that you
have more complete coverage of your project than a talk or two.

For Apache Big Data, the relevant URLs are:
Event details:
http://events.linuxfoundation.org/events/apache-big-data-europe
CFP:
http://events.linuxfoundation.org/events/apache-big-data-europe/program/cfp

For ApacheCon Europe, the relevant URLs are:
Event details: http://events.linuxfoundation.org/events/apachecon-europe
CFP: http://events.linuxfoundation.org/events/apachecon-europe/program/cfp

This year, we'll be reviewing papers "blind" - that is, looking at the
abstracts without knowing who the speaker is. This has been shown to
eliminate the "me and my buddies" nature of many tech conferences,
producing more diversity, and more new speakers. So make sure your
abstracts clearly explain what you'll be talking about.

For further updated about ApacheCon, follow us on Twitter, @ApacheCon,
or drop by our IRC channel, #apachecon on the Freenode IRC network.

-- 
Rich Bowen
WWW: http://apachecon.com/
Twitter: @ApacheCon

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5725) Efficient facets without counts for enum method

[Mikhail Khludnev (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-5725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449246#comment-15449246
 ] 

Mikhail Khludnev commented on SOLR-5725:


bq.  But I think it's bad to have facet.method otherwise have a visible effect 
on the response.

Aghhh. Got this point. This option doesn't remove counts but capping them to 1. 
What about {{facet.maxcount=1}} which will only valid with enum and allow only 
this value (1) for a while?  

> Efficient facets without counts for enum method
> ---
>
> Key: SOLR-5725
> URL: https://issues.apache.org/jira/browse/SOLR-5725
> Project: Solr
>  Issue Type: Improvement
>  Components: search
>Reporter: Alexey Kozhemiakin
>Assignee: Mikhail Khludnev
> Fix For: master (7.0), 6.3
>
> Attachments: SOLR-5725-5x.patch, SOLR-5725-master.patch, 
> SOLR-5725.patch, SOLR-5725.patch, SOLR-5725.patch, SOLR-5725.patch
>
>
> Shot version:
> This improves performance for facet.method=enum when it's enough to know that 
> facet count>0, for example when you it's when you dynamically populate 
> filters on search form. New method checks if two bitsets intersect instead of 
> counting intersection size.
> Long version:
> We have a dataset containing hundreds of millions of records, we facet by 
> dozens of fields with many of facet-excludes and have relatively small number 
> of unique values in fields, around thousands.
> Before executing search, users work with "advanced search" form, our  goal is 
> to populate dozens of filters with values which are applicable with other 
> selected values, so basically this is a use case for facets with mincount=1, 
> but without need in actual counts.
> Our performance tests showed that facet.method=enum works much better than 
> fc\fcs, probably due to a specific ratio of "docset"\"unique terms count". 
> For example average execution of query time with method fc=1500ms, fcs=2600ms 
> and with enum=280ms. Profiling indicated the majority time for enum was spent 
> on intersecting docsets.
> Hers's a patch that introduces an extension to facet calculation for 
> method=enum. Basically it uses docSetA.intersects(docSetB) instead of 
> docSetA. intersectionSize (docSetB).
> As a result we were able to reduce our average query time from 280ms to 60ms.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5725) Efficient facets without counts for enum method

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-5725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449219#comment-15449219
 ] 

David Smiley commented on SOLR-5725:


I've never heard of {{facet.zeros}}, so probably not a big concern.

bq. I'm afraid users will try to combine it with fc,fcs,uif, that's why I 
prefer to switch it as method.

I think it's okay to return an error when a particular facet.method's 
requirements aren't met, even though we don't normally do that.  But I think 
it's bad to have facet.method otherwise have a visible effect on the response.  
We already overrule what the user specifies in facet.method when we have to.  
This can be no different... set a proposed {{facet.count=false}} and then 
facet.method is ignored because we're going to effectively do that exactly one 
way.

> Efficient facets without counts for enum method
> ---
>
> Key: SOLR-5725
> URL: https://issues.apache.org/jira/browse/SOLR-5725
> Project: Solr
>  Issue Type: Improvement
>  Components: search
>Reporter: Alexey Kozhemiakin
>Assignee: Mikhail Khludnev
> Fix For: master (7.0), 6.3
>
> Attachments: SOLR-5725-5x.patch, SOLR-5725-master.patch, 
> SOLR-5725.patch, SOLR-5725.patch, SOLR-5725.patch, SOLR-5725.patch
>
>
> Shot version:
> This improves performance for facet.method=enum when it's enough to know that 
> facet count>0, for example when you it's when you dynamically populate 
> filters on search form. New method checks if two bitsets intersect instead of 
> counting intersection size.
> Long version:
> We have a dataset containing hundreds of millions of records, we facet by 
> dozens of fields with many of facet-excludes and have relatively small number 
> of unique values in fields, around thousands.
> Before executing search, users work with "advanced search" form, our  goal is 
> to populate dozens of filters with values which are applicable with other 
> selected values, so basically this is a use case for facets with mincount=1, 
> but without need in actual counts.
> Our performance tests showed that facet.method=enum works much better than 
> fc\fcs, probably due to a specific ratio of "docset"\"unique terms count". 
> For example average execution of query time with method fc=1500ms, fcs=2600ms 
> and with enum=280ms. Profiling indicated the majority time for enum was spent 
> on intersecting docsets.
> Hers's a patch that introduces an extension to facet calculation for 
> method=enum. Basically it uses docSetA.intersects(docSetB) instead of 
> docSetA. intersectionSize (docSetB).
> As a result we were able to reduce our average query time from 280ms to 60ms.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5725) Efficient facets without counts for enum method

[Mikhail Khludnev (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-5725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449184#comment-15449184
 ] 

Mikhail Khludnev commented on SOLR-5725:


* {{enumprobing}} is not the best name. Got it. {{facet.method=nonzero}} is 
good one. Don't you afraid, that user will confuse it with legacy 
{{facet.zeros}}?
bq.  there should be some parameter other than facet.method that enables this.
I'm afraid users will try to combine it with {{fc}},{{fcs}},{{uif}}, that's why 
I prefer to switch it as method. 
Thus, I like {{facet.method=exists}} from proposed ones. Btw, initially this 
name was {{intersects}}. WDYT? My point was that this method works like 
{{enum}} distinguishing from other ones, but this one is faster. What about 
{{fastenum}} or {{enumbit}}?  
   


> Efficient facets without counts for enum method
> ---
>
> Key: SOLR-5725
> URL: https://issues.apache.org/jira/browse/SOLR-5725
> Project: Solr
>  Issue Type: Improvement
>  Components: search
>Reporter: Alexey Kozhemiakin
>Assignee: Mikhail Khludnev
> Fix For: master (7.0), 6.3
>
> Attachments: SOLR-5725-5x.patch, SOLR-5725-master.patch, 
> SOLR-5725.patch, SOLR-5725.patch, SOLR-5725.patch, SOLR-5725.patch
>
>
> Shot version:
> This improves performance for facet.method=enum when it's enough to know that 
> facet count>0, for example when you it's when you dynamically populate 
> filters on search form. New method checks if two bitsets intersect instead of 
> counting intersection size.
> Long version:
> We have a dataset containing hundreds of millions of records, we facet by 
> dozens of fields with many of facet-excludes and have relatively small number 
> of unique values in fields, around thousands.
> Before executing search, users work with "advanced search" form, our  goal is 
> to populate dozens of filters with values which are applicable with other 
> selected values, so basically this is a use case for facets with mincount=1, 
> but without need in actual counts.
> Our performance tests showed that facet.method=enum works much better than 
> fc\fcs, probably due to a specific ratio of "docset"\"unique terms count". 
> For example average execution of query time with method fc=1500ms, fcs=2600ms 
> and with enum=280ms. Profiling indicated the majority time for enum was spent 
> on intersecting docsets.
> Hers's a patch that introduces an extension to facet calculation for 
> method=enum. Basically it uses docSetA.intersects(docSetB) instead of 
> docSetA. intersectionSize (docSetB).
> As a result we were able to reduce our average query time from 280ms to 60ms.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7407) Explore switching doc values to an iterator API

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449180#comment-15449180
 ] 

David Smiley commented on LUCENE-7407:
--

Hi; is this API change only at the Codec level or is it higher level as well?

> Explore switching doc values to an iterator API
> ---
>
> Key: LUCENE-7407
> URL: https://issues.apache.org/jira/browse/LUCENE-7407
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Michael McCandless
>Assignee: Michael McCandless
>  Labels: docValues
>
> I think it could be compelling if we restricted doc values to use an
> iterator API at read time, instead of the more general random access
> API we have today:
>   * It would make doc values disk usage more of a "you pay for what
> what you actually use", like postings, which is a compelling
> reduction for sparse usage.
>   * I think codecs could compress better and maybe speed up decoding
> of doc values, even in the non-sparse case, since the read-time
> API is more restrictive "forward only" instead of random access.
>   * We could remove {{getDocsWithField}} entirely, since that's
> implicit in the iteration, and the awkward "return 0 if the
> document didn't have this field" would go away.
>   * We can remove the annoying thread locals we must make today in
> {{CodecReader}}, and close the trappy "I accidentally shared a
> single XXXDocValues instance across threads", since an iterator is
> inherently "use once".
>   * We could maybe leverage the numerous optimizations we've done for
> postings over time, since the two problems ("iterate over doc ids
> and store something interesting for each") are very similar.
> This idea has come up many in the past, e.g. LUCENE-7253 is a recent
> example, and very early iterations of doc values started with exactly
> this ;)
> However, it's a truly enormous change, likely 7.0 only.  Or maybe we
> could have the new iterator APIs also ported to 6.x side by side with
> the deprecate existing random-access APIs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [CONF] Apache Solr Reference Guide > Faceting

[David Smiley]

Mikhail:
That feature is used _internally_ by the facet module, particularly for
facet refinement.  I am not sure we should advertise this feature... I
guess it's fine?  Admittedly I've used it years ago when I wanted to facet
on a bunch of pre-selected values.

On Tue, Aug 30, 2016 at 4:20 AM Mikhail Khludnev (Confluence) <
conflue...@apache.org> wrote:

> [image: avatar_ced8c191168b3bc6af3da92384625042.png]
> 
>  Mikhail
> Khludnev *edited* a page
>
> *Change comment:* i found facet.field={!terms=a,b,c}foo support in code,
> but haven't found any mentions in doc, nor even tests. I checked that it
> works, and decided to opt wiki. wdy?
> [image: page-icon.png]
> 
> Faceting
> 
>
> As described in the section Overview of Searching in Solr
> ,
> faceting is the arrangement of search results into categories based on
> indexed terms. Searchers are presented with the indexed terms, along with
> numerical counts of how many matching documents were found were each term.
> Faceting makes it easy for users to explore search results, narrowing in on
> exactly the results they are looking for.
> Panel
>
> Topics covered in this section:
> Table of Contents
> maxLevel 2
> General Parameters
>
> The table below summarizes the general parameters for controlling faceting.
>
> Parameter
>
> Description
>
> facet
> 
>
> If set to true, enables faceting.
>
> facet.query
> 
>
> Specifies a Lucene query to generate a facet count.
>
> These parameters are described in the sections below.
> The facet Parameter
>
> If set to "true," this parameter enables facet counts in the query
> response. If set to "false" to a blank or missing value, this parameter
> disables faceting. None of the other parameters listed below will have any
> effect unless this parameter is set to "true." The default value is blank.
> The facet.query Parameter
>
> This parameter allows you to specify an arbitrary query in the Lucene
> default syntax to generate a facet count. By default, Solr's faceting
> feature automatically determines the unique terms for a field and returns a
> count for each of those terms. Using facet.query, you can override this
> default behavior and select exactly which terms or expressions you would
> like to see counted. In a typical implementation of faceting, you will
> specify a number of facet.query parameters. This parameter can be
> particularly useful for numeric-range-based facets or prefix-based facets.
>
> You can set the facet.query parameter multiple times to indicate that
> multiple queries should be used as separate facet constraints.
>
> To use facet queries in a syntax other than the default syntax, prefix the
> facet query with the name of the query notation. For example, to use the
> hypothetical myfunc query parser, you could set the facet.query parameter
> like so:
>
> facet.query={!myfunc}name~fred
> Field-Value Faceting Parameters
>
> Several parameters can be used to trigger faceting based on the indexed
> terms in a field.
>
> When using this parameter, it is important to remember that "term" is a
> very specific concept in Lucene: it relates to the literal field/value
> pairs that are indexed after any analysis occurs. For text fields that
> include stemming, lowercasing, or word splitting, the resulting terms may
> not be what you expect. If you want Solr to perform both analysis (for
> searching) and faceting on the full literal strings, use the copyField
> directive in your Schema to create two versions of the field: one Text and
> one String. Make sure both are indexed="true". (For more information
> about the copyField directive, see Documents, Fields, and Schema Design
> 
> .)
>
> The table below summarizes Solr's field value faceting parameters.
>
>
>
> Parameter
>
> Description
>
> facet.field
> 
>
> Identifies a field to be treated as a facet.
>
> facet.prefix
> 
>
> Limits the terms used for faceting to those that 

[jira] [Commented] (SOLR-5725) Efficient facets without counts for enum method

[Mikhail Khludnev (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-5725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449131#comment-15449131
 ] 

Mikhail Khludnev commented on SOLR-5725:


yep. JSON Facets, is the next aim for sure (as well as query facet). fwiw, for 
now this optimization works for both {{facet.sort}} options. 

> Efficient facets without counts for enum method
> ---
>
> Key: SOLR-5725
> URL: https://issues.apache.org/jira/browse/SOLR-5725
> Project: Solr
>  Issue Type: Improvement
>  Components: search
>Reporter: Alexey Kozhemiakin
>Assignee: Mikhail Khludnev
> Fix For: master (7.0), 6.3
>
> Attachments: SOLR-5725-5x.patch, SOLR-5725-master.patch, 
> SOLR-5725.patch, SOLR-5725.patch, SOLR-5725.patch, SOLR-5725.patch
>
>
> Shot version:
> This improves performance for facet.method=enum when it's enough to know that 
> facet count>0, for example when you it's when you dynamically populate 
> filters on search form. New method checks if two bitsets intersect instead of 
> counting intersection size.
> Long version:
> We have a dataset containing hundreds of millions of records, we facet by 
> dozens of fields with many of facet-excludes and have relatively small number 
> of unique values in fields, around thousands.
> Before executing search, users work with "advanced search" form, our  goal is 
> to populate dozens of filters with values which are applicable with other 
> selected values, so basically this is a use case for facets with mincount=1, 
> but without need in actual counts.
> Our performance tests showed that facet.method=enum works much better than 
> fc\fcs, probably due to a specific ratio of "docset"\"unique terms count". 
> For example average execution of query time with method fc=1500ms, fcs=2600ms 
> and with enum=280ms. Profiling indicated the majority time for enum was spent 
> on intersecting docsets.
> Hers's a patch that introduces an extension to facet calculation for 
> method=enum. Basically it uses docSetA.intersects(docSetB) instead of 
> docSetA. intersectionSize (docSetB).
> As a result we were able to reduce our average query time from 280ms to 60ms.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9444) Fix path usage for cloud backup/restore

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449122#comment-15449122
 ] 

Uwe Schindler commented on SOLR-9444:
-

SOLR-9460, not sure which one caused this. But you made changes on the failing 
test!

> Fix path usage for cloud backup/restore
> ---
>
> Key: SOLR-9444
> URL: https://issues.apache.org/jira/browse/SOLR-9444
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Varun Thacker
> Attachments: SOLR-9444.patch
>
>
> As noted by Uwe on 
> https://issues.apache.org/jira/browse/SOLR-9242?focusedCommentId=15438925=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15438925
>  the usage of URI#getPath is wrong. 
> Creating a Jira to track this better. More details to follow



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5725) Efficient facets without counts for enum method

[Mikhail Khludnev (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-5725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449120#comment-15449120
 ] 

Mikhail Khludnev commented on SOLR-5725:


Right. There is no change in format every included count is set to "1" or "0" 
(on mincount=0)

> Efficient facets without counts for enum method
> ---
>
> Key: SOLR-5725
> URL: https://issues.apache.org/jira/browse/SOLR-5725
> Project: Solr
>  Issue Type: Improvement
>  Components: search
>Reporter: Alexey Kozhemiakin
>Assignee: Mikhail Khludnev
> Fix For: master (7.0), 6.3
>
> Attachments: SOLR-5725-5x.patch, SOLR-5725-master.patch, 
> SOLR-5725.patch, SOLR-5725.patch, SOLR-5725.patch, SOLR-5725.patch
>
>
> Shot version:
> This improves performance for facet.method=enum when it's enough to know that 
> facet count>0, for example when you it's when you dynamically populate 
> filters on search form. New method checks if two bitsets intersect instead of 
> counting intersection size.
> Long version:
> We have a dataset containing hundreds of millions of records, we facet by 
> dozens of fields with many of facet-excludes and have relatively small number 
> of unique values in fields, around thousands.
> Before executing search, users work with "advanced search" form, our  goal is 
> to populate dozens of filters with values which are applicable with other 
> selected values, so basically this is a use case for facets with mincount=1, 
> but without need in actual counts.
> Our performance tests showed that facet.method=enum works much better than 
> fc\fcs, probably due to a specific ratio of "docset"\"unique terms count". 
> For example average execution of query time with method fc=1500ms, fcs=2600ms 
> and with enum=280ms. Profiling indicated the majority time for enum was spent 
> on intersecting docsets.
> Hers's a patch that introduces an extension to facet calculation for 
> method=enum. Basically it uses docSetA.intersects(docSetB) instead of 
> docSetA. intersectionSize (docSetB).
> As a result we were able to reduce our average query time from 280ms to 60ms.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9444) Fix path usage for cloud backup/restore

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449119#comment-15449119
 ] 

Uwe Schindler commented on SOLR-9444:
-

SOLR-9460, not sure which one caused this. But you made changes on the failing 
test!

> Fix path usage for cloud backup/restore
> ---
>
> Key: SOLR-9444
> URL: https://issues.apache.org/jira/browse/SOLR-9444
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Varun Thacker
> Attachments: SOLR-9444.patch
>
>
> As noted by Uwe on 
> https://issues.apache.org/jira/browse/SOLR-9242?focusedCommentId=15438925=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15438925
>  the usage of URI#getPath is wrong. 
> Creating a Jira to track this better. More details to follow



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5725) Efficient facets without counts for enum method

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-5725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449079#comment-15449079
 ] 

David Smiley commented on SOLR-5725:


+1 to facet.method=nonzero.  Yonik's other suggestions are fine to me too.  I 
hate "enumprobing".

But I think there should be some parameter other than facet.method that enables 
this.  facet.count=false?  facet.exists=true?  I see facet.method generally as 
a _hint_.  Indeed it is a hint in JSON Faceting API (except for method=stream 
and I'm fixing that in SOLR-9142).  I don't think it should control what the 
output is in any way.

> Efficient facets without counts for enum method
> ---
>
> Key: SOLR-5725
> URL: https://issues.apache.org/jira/browse/SOLR-5725
> Project: Solr
>  Issue Type: Improvement
>  Components: search
>Reporter: Alexey Kozhemiakin
>Assignee: Mikhail Khludnev
> Fix For: master (7.0), 6.3
>
> Attachments: SOLR-5725-5x.patch, SOLR-5725-master.patch, 
> SOLR-5725.patch, SOLR-5725.patch, SOLR-5725.patch, SOLR-5725.patch
>
>
> Shot version:
> This improves performance for facet.method=enum when it's enough to know that 
> facet count>0, for example when you it's when you dynamically populate 
> filters on search form. New method checks if two bitsets intersect instead of 
> counting intersection size.
> Long version:
> We have a dataset containing hundreds of millions of records, we facet by 
> dozens of fields with many of facet-excludes and have relatively small number 
> of unique values in fields, around thousands.
> Before executing search, users work with "advanced search" form, our  goal is 
> to populate dozens of filters with values which are applicable with other 
> selected values, so basically this is a use case for facets with mincount=1, 
> but without need in actual counts.
> Our performance tests showed that facet.method=enum works much better than 
> fc\fcs, probably due to a specific ratio of "docset"\"unique terms count". 
> For example average execution of query time with method fc=1500ms, fcs=2600ms 
> and with enum=280ms. Profiling indicated the majority time for enum was spent 
> on intersecting docsets.
> Hers's a patch that introduces an extension to facet calculation for 
> method=enum. Basically it uses docSetA.intersects(docSetB) instead of 
> docSetA. intersectionSize (docSetB).
> As a result we were able to reduce our average query time from 280ms to 60ms.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Closed] (SOLR-9457) Solr node does not stop on Windows 8.1

[JIRA]

 [ 
https://issues.apache.org/jira/browse/SOLR-9457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Høydahl closed SOLR-9457.
-
Resolution: Invalid

Closing this issue as Invalid. This JIRA system is for reporting actual bugs, 
not for asking questions. Please send your question to the 
solr-u...@lucene.apache.org mailing list, see 
http://lucene.apache.org/solr/resources.html#community

Btw: You can try {{bin\solr stop -p 8983}}

> Solr node does not stop on Windows 8.1
> --
>
> Key: SOLR-9457
> URL: https://issues.apache.org/jira/browse/SOLR-9457
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Intekhab
>
> Hi,
> I am getting started with Apache Solr v6.2.0 on Windows 8.1.
> I downloaded the zip file and simply extracted the files. After navigating to 
> the extracted folder, I ran command to start the node
> 'bin\solr start'. It started the node and I could access 
> http://localhost:8983/solr/#/. After running the stop command 'bin\solr stop 
> -all', I could still access the admin page. The result of stop command shows 
> that there are no running Solr nodes. I tried the command using the Command 
> Prompt with Admin rights, but to no avail.
> Am I missing something ?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5725) Efficient facets without counts for enum method

[Yonik Seeley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-5725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449060#comment-15449060
 ] 

Yonik Seeley commented on SOLR-5725:


Thinking about how we would expose this in the JSON Facet API, I suppose it 
would be
method=exists (or "matches", or whatever name we pick) in a facet field 
block... and the output format would be the same (but counts would not be 
accurate).
It would only be an execution hint and would be ignored if not applicable (e.g. 
if one is sorting by something else and the optimization no longer makes sense).

> Efficient facets without counts for enum method
> ---
>
> Key: SOLR-5725
> URL: https://issues.apache.org/jira/browse/SOLR-5725
> Project: Solr
>  Issue Type: Improvement
>  Components: search
>Reporter: Alexey Kozhemiakin
>Assignee: Mikhail Khludnev
> Fix For: master (7.0), 6.3
>
> Attachments: SOLR-5725-5x.patch, SOLR-5725-master.patch, 
> SOLR-5725.patch, SOLR-5725.patch, SOLR-5725.patch, SOLR-5725.patch
>
>
> Shot version:
> This improves performance for facet.method=enum when it's enough to know that 
> facet count>0, for example when you it's when you dynamically populate 
> filters on search form. New method checks if two bitsets intersect instead of 
> counting intersection size.
> Long version:
> We have a dataset containing hundreds of millions of records, we facet by 
> dozens of fields with many of facet-excludes and have relatively small number 
> of unique values in fields, around thousands.
> Before executing search, users work with "advanced search" form, our  goal is 
> to populate dozens of filters with values which are applicable with other 
> selected values, so basically this is a use case for facets with mincount=1, 
> but without need in actual counts.
> Our performance tests showed that facet.method=enum works much better than 
> fc\fcs, probably due to a specific ratio of "docset"\"unique terms count". 
> For example average execution of query time with method fc=1500ms, fcs=2600ms 
> and with enum=280ms. Profiling indicated the majority time for enum was spent 
> on intersecting docsets.
> Hers's a patch that introduces an extension to facet calculation for 
> method=enum. Basically it uses docSetA.intersects(docSetB) instead of 
> docSetA. intersectionSize (docSetB).
> As a result we were able to reduce our average query time from 280ms to 60ms.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7430) Geo3d test failure

[Karl Wright (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449022#comment-15449022
 ] 

Karl Wright commented on LUCENE-7430:
-

[~mikemccand]: any ideas welcome...

> Geo3d test failure
> --
>
> Key: LUCENE-7430
> URL: https://issues.apache.org/jira/browse/LUCENE-7430
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Karl Wright
>Assignee: Karl Wright
>
> Here's the error msg:
> {code}
>[junit4] Suite: org.apache.lucene.spatial3d.TestGeo3DPoint
>[junit4] IGNOR/A 0.02s J0 | TestGeo3DPoint.testRandomBig
>[junit4]> Assumption #1: 'nightly' test group is disabled (@Nightly())
>[junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestGeo3DPoint 
> -Dtests.method=testRandomMedium -Dtests.seed=B167D2C466D257A0 
> -Dtests.slow=true -Dtests.locale=pt -Dtests.timezone=Etc/GMT+5 
> -Dtests.asserts=true -Dtests.file.encoding=Cp1252
>[junit4] FAILURE 4.06s J0 | TestGeo3DPoint.testRandomMedium <<<
>[junit4]> Throwable #1: java.lang.AssertionError: FAIL: id=4546 should 
> have matched but did not
>[junit4]>   shape=GeoWideNorthRectangle: 
> {planetmodel=PlanetModel.WGS84, 
> bottomlat=1.5707949696510948(89.2224138796), 
> leftlon=3.141592653589793(180.0), 
> rightlon=3.1140200112375265(178.42020396319145)}
>[junit4]>   bounds=XYZBounds: [xmin=-1.3551580601679786E-6 
> xmax=1.3551580601679786E-6 ymin=-1.3551580601679786E-6 
> ymax=1.3551580601679786E-6 zmin=0.9977622910211923 zmax=0.9977622930221051]
>[junit4]>   world bounds=( minX=-1.0011188539924791 
> maxX=1.0011188539924791 minY=-1.0011188539924791 maxY=1.0011188539924791 
> minZ=-0.9977622920221051 maxZ=0.9977622920221051
>[junit4]>   quantized point=[X=7.323492821176281E-6, 
> Y=-2.3309121299774915E-10, Z=0.9977622921205215] within shape? true within 
> bounds? false
>[junit4]>   unquantized point=[lat=1.570788986986606, 
> lon=-1.0056566715747591E-117([X=7.323383942914017E-6, 
> Y=-7.364809920694947E-123, Z=0.9977622919954089])] within shape? false within 
> bounds? false
>[junit4]>   docID=4438 deleted?=false
>[junit4]>   query=PointInGeo3DShapeQuery: field=point: Shape: 
> GeoWideNorthRectangle: {planetmodel=PlanetModel.WGS84, 
> bottomlat=1.5707949696510948(89.2224138796), 
> leftlon=3.141592653589793(180.0), 
> rightlon=3.1140200112375265(178.42020396319145)}
> ...
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9461) DELETENODE, REPLACENODE should pass down the 'async' param to subcommands

[Noble Paul (JIRA)]

Noble Paul created SOLR-9461:


 Summary: DELETENODE, REPLACENODE should pass down the 'async' 
param to subcommands 
 Key: SOLR-9461
 URL: https://issues.apache.org/jira/browse/SOLR-9461
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Noble Paul
Assignee: Noble Paul


The {{async}} param is used to make async calls to core admin



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9444) Fix path usage for cloud backup/restore

[Noble Paul (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449012#comment-15449012
 ] 

Noble Paul commented on SOLR-9444:
--

Which ticket exactly?

> Fix path usage for cloud backup/restore
> ---
>
> Key: SOLR-9444
> URL: https://issues.apache.org/jira/browse/SOLR-9444
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Varun Thacker
> Attachments: SOLR-9444.patch
>
>
> As noted by Uwe on 
> https://issues.apache.org/jira/browse/SOLR-9242?focusedCommentId=15438925=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15438925
>  the usage of URI#getPath is wrong. 
> Creating a Jira to track this better. More details to follow



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9444) Fix path usage for cloud backup/restore

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15448997#comment-15448997
 ] 

Uwe Schindler commented on SOLR-9444:
-

Hi,
to me this looks fine. I agree, adding a separate test for the local fs that 
tests both variants to use the path (as URI or plain path) should be added, too.
On Windows with whitespace in file names I see no errors.

It is currently only that some unrelated tests fail all the time, because 
[~noble.paul] broke Hadoop on Windows. In general Jenkins is very unstable on 
master, no successful runs since days.

> Fix path usage for cloud backup/restore
> ---
>
> Key: SOLR-9444
> URL: https://issues.apache.org/jira/browse/SOLR-9444
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Varun Thacker
> Attachments: SOLR-9444.patch
>
>
> As noted by Uwe on 
> https://issues.apache.org/jira/browse/SOLR-9242?focusedCommentId=15438925=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15438925
>  the usage of URI#getPath is wrong. 
> Creating a Jira to track this better. More details to follow



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Status of Solr Ref Guide for 6.2

[Varun Thacker]

Hi Cassandra,

Uwe's already made the changes .


Regarding the RC:
I had a doubt when I was documenting SOLR-9038 so I'll wait to hear back on
the Jira. Hopefully I can wrap that up soon and then cut an RC today

On Tue, Aug 30, 2016 at 6:44 PM, Cassandra Targett 
wrote:

> Hey Varun,
>
> You may want to email Uwe directly without cc'ing the list. He's said
> in the past that he missed the requests because of his email
> filtering, and it's better to email him directly so it doesn't get
> lost.
>
> On Tue, Aug 30, 2016 at 3:45 AM, Varun Thacker  wrote:
> > Hi Uwe,
> >
> > Could you please update the CWIKI Javadoc macro links to point to the
> 6_2_0
> > paths.
> >
> > I'll start the release process for the ref guide soon afterwards.
> >
> > On Tue, Aug 30, 2016 at 2:06 AM, Joel Bernstein 
> wrote:
> >>
> >> Hi Varun,
> >>
> >> I didn't get a chance yet to document the new streaming expressions. I'm
> >> on vacation this week. I'm planning on updating the docs early next
> week. If
> >> the docs release before then, people can read about the new streaming
> >> expressions online.
> >>
> >>
> >> On Aug 29, 2016 10:13 AM, "Varun Thacker"  wrote:
> >>>
> >>> Hi Everyone,
> >>>
> >>> I think the majority of the changes are in place. I will try to make
> the
> >>> necessary changes required for SOLR-9187/SOLR-9038/SOLR-9243 later
> today.
> >>>
> >>> I'm not sure where in
> >>> https://cwiki.apache.org/confluence/display/solr/Using+SolrJ can we
> document
> >>> SOLR-9090  . It seems like we need to beef up the SolrJ page in
> general?
> >>>
> >>> Joel, have you been able to add the new features in streaming
> expressions
> >>> to the ref guide? I can help review it
> >>>
> >>> I will aim to create an RC tomorrow morning in IST hours.
> >>>
> >>> On Thu, Aug 25, 2016 at 12:30 PM, Varun Thacker 
> >>> wrote:
> 
>  Hi Cassandra,
> 
>  I can volunteer to be the RM for the ref guide.
> 
>  We probably won't get to all the TODOs but I think let's start working
>  on it for the next few days.
> 
>  If its fine we can cut an RC on Monday 29th August and then have the
> ref
>  guide released later in the week.
> 
>  On Thu, Aug 25, 2016 at 11:23 AM, Noble Paul 
>  wrote:
> >
> > I shall document my changes today itself
> >
> > On Thu, Aug 25, 2016 at 3:39 AM, Joel Bernstein 
> > wrote:
> > > Hi Cassandra,
> > >
> > > I'm also behind on documentation for this release and on vacation
> > > next week.
> > > But I will attempt to make progress on the docs this week and do
> some
> > > documentation while on vacation as well.
> > >
> > > Joel Bernstein
> > > http://joelsolr.blogspot.com/
> > >
> > > On Wed, Aug 24, 2016 at 5:49 PM, Cassandra Targett
> > > 
> > > wrote:
> > >>
> > >> The vote for Lucene/Solr 6.2 has passed already, but the release
> > >> process for the Ref Guide has barely started.
> > >>
> > >> I'm in an offsite meeting this week, and next week have another
> > >> project to focus on. Hoss is offline until early September.
> So...the
> > >> 2
> > >> people who normally do the majority of the work to prepare and
> > >> publish
> > >> the Ref Guide aren't available.
> > >>
> > >> I've made a rather barebones TODO list, pulled only from the New
> > >> Features section of CHANGES (other sections should be reviewed
> also,
> > >> however. I won't really have time to do much more, and can't be
> the
> > >> Release Manager right now.
> > >>
> > >> Essentially, if you care to have a 6.2 Ref Guide, I'm asking for
> > >> volunteers for any/all of the following:
> > >>
> > >> - document the changes you committed
> > >> - help document changes someone else committed
> > >> - offer to be the RM and shepherd the publication process, which
> is
> > >> fully documented
> > >>
> > >> If no one steps up to help get it published soon, it's going to be
> > >> at
> > >> least a few weeks until I'll have time to devote much time on it.
> > >>
> > >> Cassandra
> > >>
> > >>
> > >> 
> -
> > >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> > >> For additional commands, e-mail: dev-h...@lucene.apache.org
> > >>
> > >
> >
> >
> >
> > --
> > -
> > Noble Paul
> >
> > 
> -
> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> > For additional commands, e-mail: dev-h...@lucene.apache.org
> >
> 
> >>>
> >
>
> 

[jira] [Commented] (SOLR-5725) Efficient facets without counts for enum method

[Yonik Seeley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-5725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15448980#comment-15448980
 ] 

Yonik Seeley commented on SOLR-5725:


enumprobing is a strange name, and doesn't convey the fact that this is the 
first "method" of faceting that actually doesn't calculate counts.
Ideas for better names:
- facet.method=nonzero
- facet.method=exists
- facet.method=matches

I don't see any description of the proposed output format, but I'll guess that 
it's the same as normal facet.field, but every included count is set to "1"? 


> Efficient facets without counts for enum method
> ---
>
> Key: SOLR-5725
> URL: https://issues.apache.org/jira/browse/SOLR-5725
> Project: Solr
>  Issue Type: Improvement
>  Components: search
>Reporter: Alexey Kozhemiakin
>Assignee: Mikhail Khludnev
> Fix For: master (7.0), 6.3
>
> Attachments: SOLR-5725-5x.patch, SOLR-5725-master.patch, 
> SOLR-5725.patch, SOLR-5725.patch, SOLR-5725.patch, SOLR-5725.patch
>
>
> Shot version:
> This improves performance for facet.method=enum when it's enough to know that 
> facet count>0, for example when you it's when you dynamically populate 
> filters on search form. New method checks if two bitsets intersect instead of 
> counting intersection size.
> Long version:
> We have a dataset containing hundreds of millions of records, we facet by 
> dozens of fields with many of facet-excludes and have relatively small number 
> of unique values in fields, around thousands.
> Before executing search, users work with "advanced search" form, our  goal is 
> to populate dozens of filters with values which are applicable with other 
> selected values, so basically this is a use case for facets with mincount=1, 
> but without need in actual counts.
> Our performance tests showed that facet.method=enum works much better than 
> fc\fcs, probably due to a specific ratio of "docset"\"unique terms count". 
> For example average execution of query time with method fc=1500ms, fcs=2600ms 
> and with enum=280ms. Profiling indicated the majority time for enum was spent 
> on intersecting docsets.
> Hers's a patch that introduces an extension to facet calculation for 
> method=enum. Basically it uses docSetA.intersects(docSetB) instead of 
> docSetA. intersectionSize (docSetB).
> As a result we were able to reduce our average query time from 280ms to 60ms.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Status of Solr Ref Guide for 6.2

[Cassandra Targett]

Hey Varun,

You may want to email Uwe directly without cc'ing the list. He's said
in the past that he missed the requests because of his email
filtering, and it's better to email him directly so it doesn't get
lost.

On Tue, Aug 30, 2016 at 3:45 AM, Varun Thacker  wrote:
> Hi Uwe,
>
> Could you please update the CWIKI Javadoc macro links to point to the 6_2_0
> paths.
>
> I'll start the release process for the ref guide soon afterwards.
>
> On Tue, Aug 30, 2016 at 2:06 AM, Joel Bernstein  wrote:
>>
>> Hi Varun,
>>
>> I didn't get a chance yet to document the new streaming expressions. I'm
>> on vacation this week. I'm planning on updating the docs early next week. If
>> the docs release before then, people can read about the new streaming
>> expressions online.
>>
>>
>> On Aug 29, 2016 10:13 AM, "Varun Thacker"  wrote:
>>>
>>> Hi Everyone,
>>>
>>> I think the majority of the changes are in place. I will try to make the
>>> necessary changes required for SOLR-9187/SOLR-9038/SOLR-9243 later today.
>>>
>>> I'm not sure where in
>>> https://cwiki.apache.org/confluence/display/solr/Using+SolrJ can we document
>>> SOLR-9090  . It seems like we need to beef up the SolrJ page in general?
>>>
>>> Joel, have you been able to add the new features in streaming expressions
>>> to the ref guide? I can help review it
>>>
>>> I will aim to create an RC tomorrow morning in IST hours.
>>>
>>> On Thu, Aug 25, 2016 at 12:30 PM, Varun Thacker 
>>> wrote:

 Hi Cassandra,

 I can volunteer to be the RM for the ref guide.

 We probably won't get to all the TODOs but I think let's start working
 on it for the next few days.

 If its fine we can cut an RC on Monday 29th August and then have the ref
 guide released later in the week.

 On Thu, Aug 25, 2016 at 11:23 AM, Noble Paul 
 wrote:
>
> I shall document my changes today itself
>
> On Thu, Aug 25, 2016 at 3:39 AM, Joel Bernstein 
> wrote:
> > Hi Cassandra,
> >
> > I'm also behind on documentation for this release and on vacation
> > next week.
> > But I will attempt to make progress on the docs this week and do some
> > documentation while on vacation as well.
> >
> > Joel Bernstein
> > http://joelsolr.blogspot.com/
> >
> > On Wed, Aug 24, 2016 at 5:49 PM, Cassandra Targett
> > 
> > wrote:
> >>
> >> The vote for Lucene/Solr 6.2 has passed already, but the release
> >> process for the Ref Guide has barely started.
> >>
> >> I'm in an offsite meeting this week, and next week have another
> >> project to focus on. Hoss is offline until early September. So...the
> >> 2
> >> people who normally do the majority of the work to prepare and
> >> publish
> >> the Ref Guide aren't available.
> >>
> >> I've made a rather barebones TODO list, pulled only from the New
> >> Features section of CHANGES (other sections should be reviewed also,
> >> however. I won't really have time to do much more, and can't be the
> >> Release Manager right now.
> >>
> >> Essentially, if you care to have a 6.2 Ref Guide, I'm asking for
> >> volunteers for any/all of the following:
> >>
> >> - document the changes you committed
> >> - help document changes someone else committed
> >> - offer to be the RM and shepherd the publication process, which is
> >> fully documented
> >>
> >> If no one steps up to help get it published soon, it's going to be
> >> at
> >> least a few weeks until I'll have time to devote much time on it.
> >>
> >> Cassandra
> >>
> >>
> >> -
> >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> >> For additional commands, e-mail: dev-h...@lucene.apache.org
> >>
> >
>
>
>
> --
> -
> Noble Paul
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>

>>>
>

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-9439) Shard split clean up logic for older failed splits is faulty

[Shalin Shekhar Mangar (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15448710#comment-15448710
 ] 

Shalin Shekhar Mangar edited comment on SOLR-9439 at 8/30/16 1:08 PM:
--

Actually the fixes for ignoring unload failures for non-existent cores that I 
had done in this issue are not necessary if the deleteshard API can internally 
call delete replica API which does the right thing. Now that we have a parallel 
mode for the delete replica API, we can just invoke it and then clear the slice 
from the cluster state. I'll put up a patch.


was (Author: shalinmangar):
Actually all the fixes that I had done in this issue are not necessary if the 
deleteshard API can internally call delete replica API which does the right 
thing. Now that we have a parallel mode for the delete replica API, we can just 
invoke it and then clear the slice from the cluster state. I'll put up a patch.

> Shard split clean up logic for older failed splits is faulty
> 
>
> Key: SOLR-9439
> URL: https://issues.apache.org/jira/browse/SOLR-9439
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 4.10.4, 5.5.2, 6.1
>Reporter: Shalin Shekhar Mangar
>Assignee: Shalin Shekhar Mangar
> Fix For: master (7.0), 6.3
>
> Attachments: Lucene-Solr-tests-master.8015.log.gz, 
> SOLR-9439-fix-deleteshard.patch, SOLR-9439.patch, SOLR-9439.patch, 
> SOLR-9439.patch
>
>
> In case a split finds that previous sub-shards exist in construction or 
> recovery state. it tries to clean them up by invoking deleteshard API. 
> However, the clean up logic tries to invoke deleteshard on the same 
> sub-shards as many times as the requested number of sub-ranges. Such repeat 
> calls to deleteshard fail and therefore fail the entire shard split operation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9439) Shard split clean up logic for older failed splits is faulty

[Shalin Shekhar Mangar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shalin Shekhar Mangar updated SOLR-9439:

Attachment: SOLR-9439-fix-deleteshard.patch

Patch which reverts some of the changes I made earlier to track and ignore 
unload failures non-existent cores because they are no longer necessary. This 
patch changes the delete shard API to call deletereplica API for all replicas 
in parallel instead of custom delete logic.

I am going to beast this test for a bit before committing.

> Shard split clean up logic for older failed splits is faulty
> 
>
> Key: SOLR-9439
> URL: https://issues.apache.org/jira/browse/SOLR-9439
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 4.10.4, 5.5.2, 6.1
>Reporter: Shalin Shekhar Mangar
>Assignee: Shalin Shekhar Mangar
> Fix For: master (7.0), 6.3
>
> Attachments: Lucene-Solr-tests-master.8015.log.gz, 
> SOLR-9439-fix-deleteshard.patch, SOLR-9439.patch, SOLR-9439.patch, 
> SOLR-9439.patch
>
>
> In case a split finds that previous sub-shards exist in construction or 
> recovery state. it tries to clean them up by invoking deleteshard API. 
> However, the clean up logic tries to invoke deleteshard on the same 
> sub-shards as many times as the requested number of sub-ranges. Such repeat 
> calls to deleteshard fail and therefore fail the entire shard split operation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Linux (32bit/jdk1.8.0_102) - Build # 17723 - Still Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/17723/
Java: 32bit/jdk1.8.0_102 -server -XX:+UseSerialGC

3 tests failed.
FAILED:  org.apache.solr.cloud.ShardSplitTest.test

Error Message:
Wrong doc count on shard1_0. See SOLR-5309 expected:<321> but was:<322>

Stack Trace:
java.lang.AssertionError: Wrong doc count on shard1_0. See SOLR-5309 
expected:<321> but was:<322>
at 
__randomizedtesting.SeedInfo.seed([BE5F5355FFE8AD0C:360B6C8F5114C0F4]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at 
org.apache.solr.cloud.ShardSplitTest.checkDocCountsAndShardStates(ShardSplitTest.java:516)
at 
org.apache.solr.cloud.ShardSplitTest.splitByUniqueKeyTest(ShardSplitTest.java:299)
at org.apache.solr.cloud.ShardSplitTest.test(ShardSplitTest.java:84)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 818 - Still Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/818/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseG1GC

4 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.client.solrj.embedded.LargeVolumeEmbeddedTest

Error Message:
ObjectTracker found 1 object(s) that were not released!!! [TransactionLog]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 1 object(s) that were not 
released!!! [TransactionLog]
at __randomizedtesting.SeedInfo.seed([CCB29B6C0452746]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at 
org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:257)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)


FAILED:  
org.apache.solr.common.cloud.TestCollectionStateWatchers.testCanWaitForNonexistantCollection

Error Message:
waitForState was not triggered by collection creation

Stack Trace:
java.lang.AssertionError: waitForState was not triggered by collection creation
at 
__randomizedtesting.SeedInfo.seed([CCB29B6C0452746:A7EBB200AD1BE36D]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.common.cloud.TestCollectionStateWatchers.testCanWaitForNonexistantCollection(TestCollectionStateWatchers.java:207)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 

[jira] [Created] (SOLR-9460) Testcase org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation breaks on Windows

[Uwe Schindler (JIRA)]

Uwe Schindler created SOLR-9460:
---

 Summary: Testcase 
org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation breaks on Windows
 Key: SOLR-9460
 URL: https://issues.apache.org/jira/browse/SOLR-9460
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
  Components: Hadoop Integration
Reporter: Uwe Schindler
Assignee: Noble Paul


The above test breaks on windows, because it does not find some crazy 
executables.

I think it should have the assumption to respect the automatically set sysprop 
"tests.disableHdfs" (if its related to HDFS) or simply 
assumeFalse(Constants.WINDOWS).:

{noformat}
[junit4] Suite: org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation
   [junit4]   2> Creating dataDir: 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestSolrCloudWithSecureImpersonation_357B7FD39643C4D2-001\init-core-data-001
   [junit4]   2> 1213353 INFO  
(SUITE-TestSolrCloudWithSecureImpersonation-seed#[357B7FD39643C4D2]-worker) [   
 ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: 
@org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 1213678 WARN  
(SUITE-TestSolrCloudWithSecureImpersonation-seed#[357B7FD39643C4D2]-worker) [   
 ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your 
platform... using builtin-java classes where applicable
   [junit4]   2> 1213719 ERROR 
(SUITE-TestSolrCloudWithSecureImpersonation-seed#[357B7FD39643C4D2]-worker) [   
 ] o.a.h.u.Shell Failed to locate the winutils binary in the hadoop binary path
   [junit4]   2> java.io.IOException: Could not locate executable 
null\bin\winutils.exe in the Hadoop binaries.
   [junit4]   2>at 
org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:356)
   [junit4]   2>at 
org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:371)
   [junit4]   2>at org.apache.hadoop.util.Shell.(Shell.java:364)
   [junit4]   2>at 
org.apache.hadoop.util.StringUtils.(StringUtils.java:80)
   [junit4]   2>at 
org.apache.hadoop.security.Groups.parseStaticMapping(Groups.java:130)
   [junit4]   2>at 
org.apache.hadoop.security.Groups.(Groups.java:94)
   [junit4]   2>at 
org.apache.hadoop.security.Groups.(Groups.java:74)
   [junit4]   2>at 
org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation.getUsersFirstGroup(TestSolrCloudWithSecureImpersonation.java:60)
   [junit4]   2>at 
org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation.getImpersonatorSettings(TestSolrCloudWithSecureImpersonation.java:86)
   [junit4]   2>at 
org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation.startup(TestSolrCloudWithSecureImpersonation.java:99)
   [junit4]   2>at sun.reflect.NativeMethodAccessorImpl.invoke0(Native 
Method)
   [junit4]   2>at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2>at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2>at java.lang.reflect.Method.invoke(Method.java:498)
   [junit4]   2>at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
   [junit4]   2>at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
   [junit4]   2>at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
   [junit4]   2>at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2>at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4]   2>at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
   [junit4]   2>at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2>at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2>at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>   

[jira] [Created] (SOLR-9459) Upgrade dependencies

[Petar Tahchiev (JIRA)]

Petar Tahchiev created SOLR-9459:


 Summary: Upgrade dependencies
 Key: SOLR-9459
 URL: https://issues.apache.org/jira/browse/SOLR-9459
 Project: Solr
  Issue Type: Improvement
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Petar Tahchiev


Hello,

my project has more than 400 dependencies and I'm trying to ban the usage of 
{{commons-collecrtions}} and {{commons-lang}} in favor of 
{{org.apache.commons:commons-collection4}} and 
{{org.apache.commons:commons-lang3}}. Unfortunately out of the 400 dependencies 
*only* solr is still using the old {{collections}} and {{lang}} dependencies 
which are more than 6 years old.

Is there a specific reason for that? Can you please update to the latest 
versions:

http://repo1.maven.org/maven2/org/apache/commons/commons-lang3/
http://repo1.maven.org/maven2/org/apache/commons/commons-collections4/
http://repo1.maven.org/maven2/org/apache/commons/commons-configuration2/
http://repo1.maven.org/maven2/org/apache/commons/commons-io/




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5725) Efficient facets without counts for enum method

[Mikhail Khludnev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-5725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Khludnev updated SOLR-5725:
---
Attachment: SOLR-5725.patch

better distributed support with tests. I'm still considering SolrCloudTestCase.

Opinions? Do you think it's ready to go? 

> Efficient facets without counts for enum method
> ---
>
> Key: SOLR-5725
> URL: https://issues.apache.org/jira/browse/SOLR-5725
> Project: Solr
>  Issue Type: Improvement
>  Components: search
>Reporter: Alexey Kozhemiakin
>Assignee: Mikhail Khludnev
> Fix For: master (7.0), 6.3
>
> Attachments: SOLR-5725-5x.patch, SOLR-5725-master.patch, 
> SOLR-5725.patch, SOLR-5725.patch, SOLR-5725.patch, SOLR-5725.patch
>
>
> Shot version:
> This improves performance for facet.method=enum when it's enough to know that 
> facet count>0, for example when you it's when you dynamically populate 
> filters on search form. New method checks if two bitsets intersect instead of 
> counting intersection size.
> Long version:
> We have a dataset containing hundreds of millions of records, we facet by 
> dozens of fields with many of facet-excludes and have relatively small number 
> of unique values in fields, around thousands.
> Before executing search, users work with "advanced search" form, our  goal is 
> to populate dozens of filters with values which are applicable with other 
> selected values, so basically this is a use case for facets with mincount=1, 
> but without need in actual counts.
> Our performance tests showed that facet.method=enum works much better than 
> fc\fcs, probably due to a specific ratio of "docset"\"unique terms count". 
> For example average execution of query time with method fc=1500ms, fcs=2600ms 
> and with enum=280ms. Profiling indicated the majority time for enum was spent 
> on intersecting docsets.
> Hers's a patch that introduces an extension to facet calculation for 
> method=enum. Basically it uses docSetA.intersects(docSetB) instead of 
> docSetA. intersectionSize (docSetB).
> As a result we were able to reduce our average query time from 280ms to 60ms.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Issue Comment Deleted] (SOLR-8029) Modernize and standardize Solr APIs

[Noble Paul (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Noble Paul updated SOLR-8029:
-
Comment: was deleted

(was: )

> Modernize and standardize Solr APIs
> ---
>
> Key: SOLR-8029
> URL: https://issues.apache.org/jira/browse/SOLR-8029
> Project: Solr
>  Issue Type: Improvement
>Affects Versions: 6.0
>Reporter: Noble Paul
>Assignee: Noble Paul
>  Labels: API, EaseOfUse
> Fix For: 6.0
>
> Attachments: SOLR-8029.patch, SOLR-8029.patch, SOLR-8029.patch, 
> SOLR-8029.patch
>
>
> Solr APIs have organically evolved and they are sometimes inconsistent with 
> each other or not in sync with the widely followed conventions of HTTP 
> protocol. Trying to make incremental changes to make them modern is like 
> applying band-aid. So, we have done a complete rethink of what the APIs 
> should be. The most notable aspects of the API are as follows:
> The new set of APIs will be placed under a new path {{/solr2}}. The legacy 
> APIs will continue to work under the {{/solr}} path as they used to and they 
> will be eventually deprecated.
> There are 4 types of requests in the new API 
> * {{/v2//*}} : Hit a collection directly or manage 
> collections/shards/replicas 
> * {{/v2//*}} : Hit a core directly or manage cores 
> * {{/v2/cluster/*}} : Operations on cluster not pertaining to any collection 
> or core. e.g: security, overseer ops etc
> This will be released as part of a major release. Check the link given below 
> for the full specification.  Your comments are welcome
> [Solr API version 2 Specification | http://bit.ly/1JYsBMQ]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9439) Shard split clean up logic for older failed splits is faulty

[Shalin Shekhar Mangar (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15448710#comment-15448710
 ] 

Shalin Shekhar Mangar commented on SOLR-9439:
-

Actually all the fixes that I had done in this issue are not necessary if the 
deleteshard API can internally call delete replica API which does the right 
thing. Now that we have a parallel mode for the delete replica API, we can just 
invoke it and then clear the slice from the cluster state. I'll put up a patch.

> Shard split clean up logic for older failed splits is faulty
> 
>
> Key: SOLR-9439
> URL: https://issues.apache.org/jira/browse/SOLR-9439
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 4.10.4, 5.5.2, 6.1
>Reporter: Shalin Shekhar Mangar
>Assignee: Shalin Shekhar Mangar
> Fix For: master (7.0), 6.3
>
> Attachments: Lucene-Solr-tests-master.8015.log.gz, SOLR-9439.patch, 
> SOLR-9439.patch, SOLR-9439.patch
>
>
> In case a split finds that previous sub-shards exist in construction or 
> recovery state. it tries to clean them up by invoking deleteshard API. 
> However, the clean up logic tries to invoke deleteshard on the same 
> sub-shards as many times as the requested number of sub-ranges. Such repeat 
> calls to deleteshard fail and therefore fail the entire shard split operation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-EA] Lucene-Solr-master-Linux (64bit/jdk-9-ea+132) - Build # 17722 - Still Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/17722/
Java: 64bit/jdk-9-ea+132 -XX:-UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  
org.apache.solr.common.cloud.TestCollectionStateWatchers.testSimpleCollectionWatch

Error Message:


Stack Trace:
java.util.concurrent.TimeoutException
at 
__randomizedtesting.SeedInfo.seed([8D80ABBA1DA2BD97:D0BB64CA5AAF22A9]:0)
at 
org.apache.solr.common.cloud.ZkStateReader.waitForState(ZkStateReader.java:1244)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.waitForState(CloudSolrClient.java:646)
at 
org.apache.solr.common.cloud.TestCollectionStateWatchers.testSimpleCollectionWatch(TestCollectionStateWatchers.java:118)
at 
jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9-ea/Native 
Method)
at 
jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9-ea/NativeMethodAccessorImpl.java:62)
at 
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9-ea/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@9-ea/Method.java:535)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(java.base@9-ea/Thread.java:843)




Build Log:
[...truncated 13236 lines...]
   [junit4] Suite: 

[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1108 - Still unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1108/

10 tests failed.
FAILED:  
org.apache.lucene.search.TestSearcherManager.testConcurrentIndexCloseSearchAndRefresh

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([B1F0697F7D82E83F]:0)


FAILED:  junit.framework.TestSuite.org.apache.lucene.search.TestSearcherManager

Error Message:
Suite timeout exceeded (>= 720 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 720 msec).
at __randomizedtesting.SeedInfo.seed([B1F0697F7D82E83F]:0)


FAILED:  junit.framework.TestSuite.org.apache.lucene.search.TestSearcherManager

Error Message:
Captured an uncaught exception in thread: Thread[id=4271, name=Thread-3444, 
state=RUNNABLE, group=TGRP-TestSearcherManager]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught 
exception in thread: Thread[id=4271, name=Thread-3444, state=RUNNABLE, 
group=TGRP-TestSearcherManager]
Caused by: java.lang.IllegalArgumentException: Document contains at least one 
immense term in field="body" (whose UTF8 encoding is longer than the max length 
32766), all of which were skipped.  Please correct the analyzer to not produce 
such terms.  The prefix of the first immense term is: '[125, 125, 123, 123, 
123, 123, 123, 115, 117, 98, 115, 116, 99, 124, 125, 125, 125, 123, 123, 123, 
49, 125, 125, 125, 124, 123, 123, 123, 112, 49]...', original message: bytes 
can be at most 32766 in length; got 94384
at __randomizedtesting.SeedInfo.seed([B1F0697F7D82E83F]:0)
at 
org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:772)
at 
org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:417)
at 
org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:373)
at 
org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:231)
at 
org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:478)
at 
org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1562)
at 
org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1307)
at 
org.apache.lucene.search.TestSearcherManager$8.run(TestSearcherManager.java:557)
Suppressed: java.lang.IllegalStateException: close() called in wrong 
state: INCREMENT
at 
org.apache.lucene.analysis.MockTokenizer.fail(MockTokenizer.java:125)
at 
org.apache.lucene.analysis.MockTokenizer.close(MockTokenizer.java:292)
at 
org.apache.lucene.analysis.TokenFilter.close(TokenFilter.java:58)
at 
org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:788)
... 7 more
Caused by: org.apache.lucene.util.BytesRefHash$MaxBytesLengthExceededException: 
bytes can be at most 32766 in length; got 94384
at org.apache.lucene.util.BytesRefHash.add(BytesRefHash.java:263)
at 
org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:149)
at 
org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:762)
... 7 more


FAILED:  org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader

Error Message:
No live SolrServers available to handle this 
request:[http://127.0.0.1:33079/forceleader_test_collection_shard1_replica2]

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: No live 
SolrServers available to handle this 
request:[http://127.0.0.1:33079/forceleader_test_collection_shard1_replica2]
at 
__randomizedtesting.SeedInfo.seed([89FA53BCCCDAFAFA:6F6D677CF558039B]:0)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:755)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1161)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1050)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:992)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:753)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:741)
at 
org.apache.solr.cloud.ForceLeaderTest.sendDoc(ForceLeaderTest.java:424)
at 
org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader(ForceLeaderTest.java:131)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   

[jira] [Created] (SOLR-9458) DocumentDictionaryFactory StackOverflowError on many documents

[Chris de Kok (JIRA)]

Chris de Kok created SOLR-9458:
--

 Summary: DocumentDictionaryFactory StackOverflowError on many 
documents
 Key: SOLR-9458
 URL: https://issues.apache.org/jira/browse/SOLR-9458
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
  Components: Suggester
Affects Versions: 6.2, 6.1
Reporter: Chris de Kok


When using the FuzzyLookupFactory in combinarion with the 
DocumentDictionaryFactory it will throw a stackoverflow trying to build the 
dictionary.

Using the HighFrequencyDictionaryFactory works ok but behaves very different.

```


suggest
suggestions
suggestions
FuzzyLookupFactory
DocumentDictionaryFactory
suggest_fuzzy
true
false
false
true
0



null:java.lang.StackOverflowError
at 
org.apache.lucene.util.automaton.Operations.topoSortStatesRecurse(Operations.java:1311)
at 
org.apache.lucene.util.automaton.Operations.topoSortStatesRecurse(Operations.java:1311)
at 
org.apache.lucene.util.automaton.Operations.topoSortStatesRecurse(Operations.java:1311)
at 
org.apache.lucene.util.automaton.Operations.topoSortStatesRecurse(Operations.java:1311)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7407) Explore switching doc values to an iterator API

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15448637#comment-15448637
 ] 

Michael McCandless commented on LUCENE-7407:


Just a quick progress update here: I've managed to cut over 4 (Numeric, Binary, 
Sorted, norms) of the 6 DV classes ... working on SortedSet now, and then 
finally SortedNumeric.

Then I need to remove the crazy bridge classes, then remove the old 
(non-iterator) classes, then rename the iterator classes back to the original 
names.

Net/net I think can work, but I still have plenty of nocommits to sort out.

> Explore switching doc values to an iterator API
> ---
>
> Key: LUCENE-7407
> URL: https://issues.apache.org/jira/browse/LUCENE-7407
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Michael McCandless
>Assignee: Michael McCandless
>  Labels: docValues
>
> I think it could be compelling if we restricted doc values to use an
> iterator API at read time, instead of the more general random access
> API we have today:
>   * It would make doc values disk usage more of a "you pay for what
> what you actually use", like postings, which is a compelling
> reduction for sparse usage.
>   * I think codecs could compress better and maybe speed up decoding
> of doc values, even in the non-sparse case, since the read-time
> API is more restrictive "forward only" instead of random access.
>   * We could remove {{getDocsWithField}} entirely, since that's
> implicit in the iteration, and the awkward "return 0 if the
> document didn't have this field" would go away.
>   * We can remove the annoying thread locals we must make today in
> {{CodecReader}}, and close the trappy "I accidentally shared a
> single XXXDocValues instance across threads", since an iterator is
> inherently "use once".
>   * We could maybe leverage the numerous optimizations we've done for
> postings over time, since the two problems ("iterate over doc ids
> and store something interesting for each") are very similar.
> This idea has come up many in the past, e.g. LUCENE-7253 is a recent
> example, and very early iterations of doc values started with exactly
> this ;)
> However, it's a truly enormous change, likely 7.0 only.  Or maybe we
> could have the new iterator APIs also ported to 6.x side by side with
> the deprecate existing random-access APIs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 3514 - Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3514/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.update.AutoCommitTest.testMaxDocs

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
at 
__randomizedtesting.SeedInfo.seed([477BBB89945A05EB:FEFA6D56B8B00161]:0)
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:782)
at 
org.apache.solr.update.AutoCommitTest.testMaxDocs(AutoCommitTest.java:198)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: 
xpath=//result[@numFound=1]
xml response was: 

00


request was:q=id:14=standard=0=20=2.2
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:775)
... 40 more




Build Log:
[...truncated 10479 lines...]
   [junit4] Suite: org.apache.solr.update.AutoCommitTest
   

[jira] [Commented] (SOLR-8589) Add aliases to the LIST action results in the Collections API

[JIRA]

[ 
https://issues.apache.org/jira/browse/SOLR-8589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15448555#comment-15448555
 ] 

Jan Høydahl commented on SOLR-8589:
---

Before continuing on this, I'd like to check with [~markrmil...@gmail.com] and 
[~smolloy] who were working on a {{LISTALIAS}} command in SOLR-4968. Are you 
fine with closing that issue and pursuing this one?

> Add aliases to the LIST action results in the Collections API
> -
>
> Key: SOLR-8589
> URL: https://issues.apache.org/jira/browse/SOLR-8589
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Affects Versions: 5.4.1
>Reporter: Shawn Heisey
>Assignee: Shawn Heisey
>Priority: Minor
> Attachments: SOLR-8589.patch, SOLR-8589.patch, SOLR-8589.patch, 
> SOLR-8589.patch, solr-8589-new-list-details-aliases.png
>
>
> Although it is possible to get a list of SolrCloud aliases vi an HTTP API, it 
> is not available as a typical query response, I believe it is only available 
> via the http API for zookeeper.
> The results from the LIST action in the Collections API is well-situated to 
> handle this. The current results are contained in a "collections" node, we 
> can simply add an "aliases" node if there are any aliases defined.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2151) MappingUpdateProcessor

[JIRA]

[ 
https://issues.apache.org/jira/browse/SOLR-2151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15448528#comment-15448528
 ] 

Jan Høydahl commented on SOLR-2151:
---

I have updated the MappingUpdateProcessor for master and put it up on 
https://github.com/cominvent/solr-update-processors
Any interest of including it in core?

> MappingUpdateProcessor
> --
>
> Key: SOLR-2151
> URL: https://issues.apache.org/jira/browse/SOLR-2151
> Project: Solr
>  Issue Type: New Feature
>  Components: update
>Reporter: Jan Høydahl
> Attachments: SOLR-2151-solr1.4.patch, SOLR-2151-solr1.4.patch
>
>
> Implement an UpdateProcessor which can read one field, lookup the value in a 
> dictionary and write the mapped value to another field.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8589) Add aliases to the LIST action results in the Collections API

[JIRA]

[ 
https://issues.apache.org/jira/browse/SOLR-8589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15448491#comment-15448491
 ] 

Jan Høydahl commented on SOLR-8589:
---

bq. If we do this, do you think there is going to be confusion between when to 
use CollectionAdminParams.COLLECTION and CollectionAdminParams.COLLECTIONS? 
That's the only potential downside that I can see.

Thinking about it, perhaps it is better to skip this detailed control of what 
to list. Keep things simple. We add a section to the LIST XML response. It is 
generally useful, it is not huge in size, and no real benefit to be able to 
turn it on or off imo.

{quote}
bq. Also, I think we should return aliases by default.
I don't know if there is anybody out there that has some kind of monitoring 
script set up to parse the result of LIST, and in general I try to be super 
conservative about changing APIs. Maybe we default to returning them in 7.0, 
and not returning them in 6.x? That sounds like a fair compromise to me.
{quote}

This is XML, and we *add* a new XML node, not changing anything of the 
existing. Any XML parser used to retrieve the {{}} 
would still work exactly as before. So even if someones script breaks, that 
will still not be a bug on our parts, this is not a back-compat break.

I can take a stab at simplifying the patch :)

> Add aliases to the LIST action results in the Collections API
> -
>
> Key: SOLR-8589
> URL: https://issues.apache.org/jira/browse/SOLR-8589
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Affects Versions: 5.4.1
>Reporter: Shawn Heisey
>Assignee: Shawn Heisey
>Priority: Minor
> Attachments: SOLR-8589.patch, SOLR-8589.patch, SOLR-8589.patch, 
> SOLR-8589.patch, solr-8589-new-list-details-aliases.png
>
>
> Although it is possible to get a list of SolrCloud aliases vi an HTTP API, it 
> is not available as a typical query response, I believe it is only available 
> via the http API for zookeeper.
> The results from the LIST action in the Collections API is well-situated to 
> handle this. The current results are contained in a "collections" node, we 
> can simply add an "aliases" node if there are any aliases defined.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Status of Solr Ref Guide for 6.2

[Varun Thacker]

Hi Uwe,

Could you please update the CWIKI Javadoc macro links to point to the 6_2_0
paths.

I'll start the release process for the ref guide soon afterwards.

On Tue, Aug 30, 2016 at 2:06 AM, Joel Bernstein  wrote:

> Hi Varun,
>
> I didn't get a chance yet to document the new streaming expressions. I'm
> on vacation this week. I'm planning on updating the docs early next week.
> If the docs release before then, people can read about the new streaming
> expressions online.
>
> On Aug 29, 2016 10:13 AM, "Varun Thacker"  wrote:
>
>> Hi Everyone,
>>
>> I think the majority of the changes are in place. I will try to make the
>> necessary changes required for SOLR-9187/SOLR-9038/SOLR-9243 later today.
>>
>> I'm not sure where in https://cwiki.apache.org/confl
>> uence/display/solr/Using+SolrJ can we document SOLR-9090  . It seems
>> like we need to beef up the SolrJ page in general?
>>
>> Joel, have you been able to add the new features in streaming expressions
>> to the ref guide? I can help review it
>>
>> I will aim to create an RC tomorrow morning in IST hours.
>>
>> On Thu, Aug 25, 2016 at 12:30 PM, Varun Thacker 
>> wrote:
>>
>>> Hi Cassandra,
>>>
>>> I can volunteer to be the RM for the ref guide.
>>>
>>> We probably won't get to all the TODOs but I think let's start working
>>> on it for the next few days.
>>>
>>> If its fine we can cut an RC on Monday 29th August and then have the ref
>>> guide released later in the week.
>>>
>>> On Thu, Aug 25, 2016 at 11:23 AM, Noble Paul 
>>> wrote:
>>>
 I shall document my changes today itself

 On Thu, Aug 25, 2016 at 3:39 AM, Joel Bernstein 
 wrote:
 > Hi Cassandra,
 >
 > I'm also behind on documentation for this release and on vacation
 next week.
 > But I will attempt to make progress on the docs this week and do some
 > documentation while on vacation as well.
 >
 > Joel Bernstein
 > http://joelsolr.blogspot.com/
 >
 > On Wed, Aug 24, 2016 at 5:49 PM, Cassandra Targett <
 casstarg...@gmail.com>
 > wrote:
 >>
 >> The vote for Lucene/Solr 6.2 has passed already, but the release
 >> process for the Ref Guide has barely started.
 >>
 >> I'm in an offsite meeting this week, and next week have another
 >> project to focus on. Hoss is offline until early September. So...the
 2
 >> people who normally do the majority of the work to prepare and
 publish
 >> the Ref Guide aren't available.
 >>
 >> I've made a rather barebones TODO list, pulled only from the New
 >> Features section of CHANGES (other sections should be reviewed also,
 >> however. I won't really have time to do much more, and can't be the
 >> Release Manager right now.
 >>
 >> Essentially, if you care to have a 6.2 Ref Guide, I'm asking for
 >> volunteers for any/all of the following:
 >>
 >> - document the changes you committed
 >> - help document changes someone else committed
 >> - offer to be the RM and shepherd the publication process, which is
 >> fully documented
 >>
 >> If no one steps up to help get it published soon, it's going to be at
 >> least a few weeks until I'll have time to devote much time on it.
 >>
 >> Cassandra
 >>
 >> 
 -
 >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 >> For additional commands, e-mail: dev-h...@lucene.apache.org
 >>
 >



 --
 -
 Noble Paul

 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org


>>>
>>

[jira] [Created] (SOLR-9457) Solr node does not stop on Windows 8.1

[Intekhab (JIRA)]

Intekhab created SOLR-9457:
--

 Summary: Solr node does not stop on Windows 8.1
 Key: SOLR-9457
 URL: https://issues.apache.org/jira/browse/SOLR-9457
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Intekhab


Hi,

I am getting started with Apache Solr v6.2.0 on Windows 8.1.


I downloaded the zip file and simply extracted the files. After navigating to 
the extracted folder, I ran command to start the node

'bin\solr start'. It started the node and I could access 
http://localhost:8983/solr/#/. After running the stop command 'bin\solr stop 
-all', I could still access the admin page. The result of stop command shows 
that there are no running Solr nodes. I tried the command using the Command 
Prompt with Admin rights, but to no avail.

Am I missing something ?





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9374) Speed up Jmx MBean retrieval for FieldCache

[Tim Owen (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15448437#comment-15448437
 ] 

Tim Owen commented on SOLR-9374:


No problem, thanks for merging!

> Speed up Jmx MBean retrieval for FieldCache
> ---
>
> Key: SOLR-9374
> URL: https://issues.apache.org/jira/browse/SOLR-9374
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: JMX, web gui
>Affects Versions: 6.1, master (7.0)
>Reporter: Tim Owen
>Assignee: Shalin Shekhar Mangar
>Priority: Minor
> Fix For: master (7.0), 6.3
>
> Attachments: SOLR-9374.patch
>
>
> The change made in SOLR-8892 allowed for Jmx requests for MBean info to skip 
> displaying the full contents of FieldCache entries, and just return the count.
> However, it still computes all the field cache entry info but throws it away 
> and uses only the number of entries. This can make the Jmx MBean retrieval 
> quite slow which is not ideal for regular polling for monitoring purposes. 
> We've typically found the Jmx call took over 1 minute to complete, and jstack 
> output showed that building the stats for this bean was the culprit.
> With this patch, the time is much reduced, usually less than 10 seconds. The 
> response contents are unchanged.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9448) [subquery] can't pull fields from SolrCloud collection if it has a differently named uniqueKey

[Mikhail Khludnev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Khludnev updated SOLR-9448:
---
Description: 
straightforward \[subquery] implementation executes requests on a caller 
collection, but just hitting another one with 
{{caller/select?q=..=callee}}. The problem is that for 
{{GET_FIELDS}} it uses uniqKey from a caller collection but not a callee one. 
Another observation, at that case both single sharded collections are 
collocated at the same instance. Then, subquery can't be parsed if it queries a 
field which are absent in caller schema. All of this seems pretty strange like 
hitting an edge case. 

h2. workaround
Perhaps you can collocate secondary index and call it {{fromIndex=callee}}.
Or you can name uniqKey the same, keeping the different app semantic.
 

  was:
straightforward \[subquery] implementation executes requests on a caller 
collection, but just hitting another one with 
{{caller/select?q=..=callee }}. The problem is that for 
{{GET_FIELDS}} it uses uniqKey from a caller collection but not a callee one. 
Another observation, at that case both single sharded collections are 
collocated at the same instance. Then, subquery can't be parsed if it queries a 
field which are absent in caller schema. All of this seems pretty strange like 
hitting an edge case. 

h2. workaround
Perhaps you can collocate secondary index and call it {{fromIndex=callee}}.
Or you can name uniqKey the same, keeping the different app semantic.
 


> [subquery] can't pull fields from SolrCloud collection if it has a 
> differently named uniqueKey
> --
>
> Key: SOLR-9448
> URL: https://issues.apache.org/jira/browse/SOLR-9448
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 6.1, 6.2
>Reporter: Mikhail Khludnev
>Assignee: Mikhail Khludnev
>
> straightforward \[subquery] implementation executes requests on a caller 
> collection, but just hitting another one with 
> {{caller/select?q=..=callee}}. The problem is that for 
> {{GET_FIELDS}} it uses uniqKey from a caller collection but not a callee one. 
> Another observation, at that case both single sharded collections are 
> collocated at the same instance. Then, subquery can't be parsed if it queries 
> a field which are absent in caller schema. All of this seems pretty strange 
> like hitting an edge case. 
> h2. workaround
> Perhaps you can collocate secondary index and call it {{fromIndex=callee}}.
> Or you can name uniqKey the same, keeping the different app semantic.
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-EA] Lucene-Solr-master-Linux (64bit/jdk-9-ea+132) - Build # 17721 - Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/17721/
Java: 64bit/jdk-9-ea+132 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  org.apache.solr.security.BasicAuthIntegrationTest.testBasics

Error Message:
IOException occured when talking to server at: http://127.0.0.1:37939/solr

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: IOException occured when 
talking to server at: http://127.0.0.1:37939/solr
at 
__randomizedtesting.SeedInfo.seed([BFEF358408043F55:82379BA830EA6125]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:622)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:261)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:250)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:415)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:367)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1280)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1050)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:992)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at 
org.apache.solr.security.BasicAuthIntegrationTest.doExtraTests(BasicAuthIntegrationTest.java:116)
at 
org.apache.solr.cloud.TestMiniSolrCloudClusterBase.testCollectionCreateSearchDelete(TestMiniSolrCloudClusterBase.java:196)
at 
org.apache.solr.cloud.TestMiniSolrCloudClusterBase.testBasics(TestMiniSolrCloudClusterBase.java:79)
at 
jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9-ea/Native 
Method)
at 
jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9-ea/NativeMethodAccessorImpl.java:62)
at 
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9-ea/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@9-ea/Method.java:535)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 

[jira] [Commented] (SOLR-9389) HDFS Transaction logs stay open for writes which leaks Xceivers

[Tim Owen (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15448406#comment-15448406
 ] 

Tim Owen commented on SOLR-9389:


Great, thanks for reviewing and testing this Mark :)

> HDFS Transaction logs stay open for writes which leaks Xceivers
> ---
>
> Key: SOLR-9389
> URL: https://issues.apache.org/jira/browse/SOLR-9389
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Hadoop Integration, hdfs
>Affects Versions: 6.1, master (7.0)
>Reporter: Tim Owen
>Assignee: Mark Miller
> Fix For: master (7.0), 6.3
>
> Attachments: SOLR-9389.patch
>
>
> The HdfsTransactionLog implementation keeps a Hadoop FSDataOutputStream open 
> for its whole lifetime, which consumes two threads on the HDFS data node 
> server (dataXceiver and packetresponder) even once the Solr tlog has finished 
> being written to.
> This means for a cluster with many indexes on HDFS, the number of Xceivers 
> can keep growing and eventually hit the limit of 4096 on the data nodes. It's 
> especially likely for indexes that have low write rates, because Solr keeps 
> enough tlogs around to contain 100 documents (up to a limit of 10 tlogs). 
> There's also the issue that attempting to write to a finished tlog would be a 
> major bug, so closing it for writes helps catch that.
> Our cluster during testing had 100+ collections with 100 shards each, spread 
> across 8 boxes (each running 4 solr nodes and 1 hdfs data node) and with 3x 
> replication for the tlog files, this meant we hit the xceiver limit fairly 
> easily and had to use the attached patch to ensure tlogs were closed for 
> writes once finished.
> The patch introduces an extra lifecycle state for the tlog, so it can be 
> closed for writes and free up the HDFS resources, while still being available 
> for reading. I've tried to make it as unobtrusive as I could, but there's 
> probably a better way. I have not changed the behaviour of the local disk 
> tlog implementation, because it only consumes a file descriptor regardless of 
> read or write.
> nb We have decided not to use Solr-on-HDFS now, we're using local disk (for 
> various reasons). So I don't have a HDFS cluster to do further testing on 
> this, I'm just contributing the patch which worked for us.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9450) Link to online Javadocs instead of distributing with binary download

[JIRA]

 [ 
https://issues.apache.org/jira/browse/SOLR-9450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Høydahl updated SOLR-9450:
--
Attachment: SOLR-9450.patch

Attaching new patch:

* We now build the online version of docs in {{$\{javadoc-online.dir\}.}} which 
is {{build/docs-online}}, to avoid overwriting the full javadocs
* The {{create-package}} target only depends on {{documentation-online}}, while 
{{package}} now also depends on {{documentation}} target, to make sure that 
{{build/docs}} is also generated in that case.
* Reuses property {{lucene.javadoc.url}}, so no need to instrument Jenkins for 
{{solr.javadoc.url}}


> Link to online Javadocs instead of distributing with binary download
> 
>
> Key: SOLR-9450
> URL: https://issues.apache.org/jira/browse/SOLR-9450
> Project: Solr
>  Issue Type: Sub-task
>  Components: Build
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
> Fix For: master (7.0), 6.3
>
> Attachments: SOLR-9450.patch, SOLR-9450.patch
>
>
> Spinoff from SOLR-6806. This sub task will replace the contents of {{docs}} 
> in the binary download with a link to the online JavaDocs. The build should 
> make sure to generate a link to the correct version. I believe this is the 
> correct tamplate: http://lucene.apache.org/solr/6_2_0/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9038) Support snapshot management functionality for a solr collection

[Varun Thacker (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15448388#comment-15448388
 ] 

Varun Thacker commented on SOLR-9038:
-

Hi David,Hrishikesh

I was trying to document this feature in the ref guide and had a doubt:

1. If we create a snapshot , there is no way to restore a snapshot directly 
into the same index? We have to first backup the snapshot and then restore that?

> Support snapshot management functionality for a solr collection
> ---
>
> Key: SOLR-9038
> URL: https://issues.apache.org/jira/browse/SOLR-9038
> Project: Solr
>  Issue Type: New Feature
>  Components: SolrCloud
>Reporter: Hrishikesh Gadre
>Assignee: David Smiley
>
> Currently work is under-way to implement backup/restore API for Solr cloud 
> (SOLR-5750). SOLR-5750 is about providing an ability to "copy" index files 
> and collection metadata to a configurable location. 
> In addition to this, we should also provide a facility to create "named" 
> snapshots for Solr collection. Here by "snapshot" I mean configuring the 
> underlying Lucene IndexDeletionPolicy to not delete a specific commit point 
> (e.g. using PersistentSnapshotIndexDeletionPolicy). This should not be 
> confused with SOLR-5340 which implements core level "backup" functionality.
> The primary motivation of this feature is to decouple recording/preserving a 
> known consistent state of a collection from actually "copying" the relevant 
> files to a physically separate location. This decoupling have number of 
> advantages
> - We can use specialized data-copying tools for transferring Solr index 
> files. e.g. in Hadoop environment, typically 
> [distcp|https://hadoop.apache.org/docs/r1.2.1/distcp2.html] tool is used to 
> copy files from one location to other. This tool provides various options to 
> configure degree of parallelism, bandwidth usage as well as integration with 
> different types and versions of file systems (e.g. AWS S3, Azure Blob store 
> etc.)
> - This separation of concern would also help Solr to focus on the key 
> functionality (i.e. querying and indexing) while delegating the copy 
> operation to the tools built for that purpose.
> - Users can decide if/when to copy the data files as against creating a 
> snapshot. e.g. a user may want to create a snapshot of a collection before 
> making an experimental change (e.g. updating/deleting docs, schema change 
> etc.). If the experiment is successful, he can delete the snapshot (without 
> having to copy the files). If the experiment is failed, then he can copy the 
> files associated with the snapshot and restore.
> Note that Apache Blur project is also providing a similar feature 
> [BLUR-132|https://issues.apache.org/jira/browse/BLUR-132]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-6.x-Windows (32bit/jdk1.8.0_102) - Build # 423 - Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Windows/423/
Java: 32bit/jdk1.8.0_102 -server -XX:+UseG1GC

1 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.schema.TestManagedSchemaAPI

Error Message:
ObjectTracker found 10 object(s) that were not released!!! [TransactionLog, 
MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, 
TransactionLog, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, 
MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 10 object(s) that were not 
released!!! [TransactionLog, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, 
MockDirectoryWrapper, TransactionLog, MockDirectoryWrapper, 
MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, 
MockDirectoryWrapper]
at __randomizedtesting.SeedInfo.seed([5768C2F8BF6D20EF]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at 
org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:258)
at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 12491 lines...]
   [junit4] Suite: org.apache.solr.schema.TestManagedSchemaAPI
   [junit4]   2> Creating dataDir: 
C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\solr\build\solr-core\test\J1\temp\solr.schema.TestManagedSchemaAPI_5768C2F8BF6D20EF-001\init-core-data-001
   [junit4]   2> 2845041 INFO  
(SUITE-TestManagedSchemaAPI-seed#[5768C2F8BF6D20EF]-worker) [] 
o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: 
@org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 2845043 INFO  
(SUITE-TestManagedSchemaAPI-seed#[5768C2F8BF6D20EF]-worker) [] 
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2845043 INFO  (Thread-6050) [] o.a.s.c.ZkTestServer client 
port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2845043 INFO  (Thread-6050) [] o.a.s.c.ZkTestServer 
Starting server
   [junit4]   2> 2845144 INFO  
(SUITE-TestManagedSchemaAPI-seed#[5768C2F8BF6D20EF]-worker) [] 
o.a.s.c.ZkTestServer start zk server on port:61539
   [junit4]   2> 2845144 INFO  
(SUITE-TestManagedSchemaAPI-seed#[5768C2F8BF6D20EF]-worker) [] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2845145 INFO  
(SUITE-TestManagedSchemaAPI-seed#[5768C2F8BF6D20EF]-worker) [] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2845158 INFO  (zkCallback-11286-thread-1) [] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@20bc0c name:ZooKeeperConnection 

[jira] [Commented] (SOLR-9444) Fix path usage for cloud backup/restore

[Varun Thacker (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15448315#comment-15448315
 ] 

Varun Thacker commented on SOLR-9444:
-

bq.  will try it out in a minute on my windows with whitespace in path.

I was thinking of maybe adding this as part of 
{{TestLocalFSCloudBackupRestore#getBackupLocation}} also?

> Fix path usage for cloud backup/restore
> ---
>
> Key: SOLR-9444
> URL: https://issues.apache.org/jira/browse/SOLR-9444
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Varun Thacker
> Attachments: SOLR-9444.patch
>
>
> As noted by Uwe on 
> https://issues.apache.org/jira/browse/SOLR-9242?focusedCommentId=15438925=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15438925
>  the usage of URI#getPath is wrong. 
> Creating a Jira to track this better. More details to follow



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-9444) Fix path usage for cloud backup/restore

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15448306#comment-15448306
 ] 

Uwe Schindler edited comment on SOLR-9444 at 8/30/16 7:21 AM:
--

In the future, once the HDFS people created a working filesystem for Java 7, we 
may remove the HDFS special case, but for now its good to have. FYI: Using this 
new impl, it should be possible to create a backup in a zip file using the ZIP 
file system of Java 7+: 
http://docs.oracle.com/javase/8/docs/technotes/guides/io/fsp/zipfilesystemprovider.html
 (see second example)


was (Author: thetaphi):
In the future, once the HDFS people created a working filesystem for Java 7, we 
may remove the HDFS special case, but for now its good to have.

> Fix path usage for cloud backup/restore
> ---
>
> Key: SOLR-9444
> URL: https://issues.apache.org/jira/browse/SOLR-9444
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Varun Thacker
> Attachments: SOLR-9444.patch
>
>
> As noted by Uwe on 
> https://issues.apache.org/jira/browse/SOLR-9242?focusedCommentId=15438925=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15438925
>  the usage of URI#getPath is wrong. 
> Creating a Jira to track this better. More details to follow



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9444) Fix path usage for cloud backup/restore

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15448306#comment-15448306
 ] 

Uwe Schindler commented on SOLR-9444:
-

In the future, once the HDFS people created a working filesystem for Java 7, we 
may remove the HDFS special case, but for now its good to have.

> Fix path usage for cloud backup/restore
> ---
>
> Key: SOLR-9444
> URL: https://issues.apache.org/jira/browse/SOLR-9444
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Varun Thacker
> Attachments: SOLR-9444.patch
>
>
> As noted by Uwe on 
> https://issues.apache.org/jira/browse/SOLR-9242?focusedCommentId=15438925=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15438925
>  the usage of URI#getPath is wrong. 
> Creating a Jira to track this better. More details to follow



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9444) Fix path usage for cloud backup/restore

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15448302#comment-15448302
 ] 

Uwe Schindler commented on SOLR-9444:
-

Hi,
this looks cool! I will try it out in a minute on my windows with whitespace in 
path.
This looks exactly as I proposed:
- Only pass URI instances around instead of strings
- Change createURI to take the base URI as URI
- Add a new createURI that takes a single string and is responsible to create 
aplain RI from a string, which is repository specific

Maybe rename the createURI that takes a baseURI and componenets to have a 
different name? E.g. resolveURI?

> Fix path usage for cloud backup/restore
> ---
>
> Key: SOLR-9444
> URL: https://issues.apache.org/jira/browse/SOLR-9444
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Varun Thacker
> Attachments: SOLR-9444.patch
>
>
> As noted by Uwe on 
> https://issues.apache.org/jira/browse/SOLR-9242?focusedCommentId=15438925=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15438925
>  the usage of URI#getPath is wrong. 
> Creating a Jira to track this better. More details to follow



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] lucene-solr pull request #74: [SOLR-9444] Fix path usage for cloud backup/re...

[uschindler]

Github user uschindler commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/74#discussion_r76740913
  
--- Diff: 
solr/core/src/java/org/apache/solr/core/backup/repository/LocalFileSystemRepository.java
 ---
@@ -58,21 +59,28 @@ public void init(NamedList args) {
   }
 
   @Override
-  public URI createURI(String... pathComponents) {
-Preconditions.checkArgument(pathComponents.length > 0);
-
-String basePath = Preconditions.checkNotNull(pathComponents[0]);
-// Note the URI.getPath() invocation on Windows platform generates an 
invalid URI.
-// Refer to 
http://stackoverflow.com/questions/9834776/java-nio-file-path-issue
-// Since the caller may have used this method to generate the string 
representation
-// for the pathComponents, we implement a work-around specifically for 
Windows platform
-// to remove the leading '/' character.
-if (Constants.WINDOWS) {
-  basePath = basePath.replaceFirst("^/(.:/)", "$1");
+  public URI createURI(String location) {
+Preconditions.checkNotNull(location);
+
+URI result = null;
+try {
--- End diff --

Nice. This is exactly as I proposed. So people can use both URIs with a 
file: or just a plain path. URI.isAbsolute() returns false, if scheme ("file:") 
is missing: 
 "A 
URI is absolute if, and only if, it has a scheme component."


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9444) Fix path usage for cloud backup/restore

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15448293#comment-15448293
 ] 

ASF GitHub Bot commented on SOLR-9444:
--

Github user uschindler commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/74#discussion_r76740913
  
--- Diff: 
solr/core/src/java/org/apache/solr/core/backup/repository/LocalFileSystemRepository.java
 ---
@@ -58,21 +59,28 @@ public void init(NamedList args) {
   }
 
   @Override
-  public URI createURI(String... pathComponents) {
-Preconditions.checkArgument(pathComponents.length > 0);
-
-String basePath = Preconditions.checkNotNull(pathComponents[0]);
-// Note the URI.getPath() invocation on Windows platform generates an 
invalid URI.
-// Refer to 
http://stackoverflow.com/questions/9834776/java-nio-file-path-issue
-// Since the caller may have used this method to generate the string 
representation
-// for the pathComponents, we implement a work-around specifically for 
Windows platform
-// to remove the leading '/' character.
-if (Constants.WINDOWS) {
-  basePath = basePath.replaceFirst("^/(.:/)", "$1");
+  public URI createURI(String location) {
+Preconditions.checkNotNull(location);
+
+URI result = null;
+try {
--- End diff --

Nice. This is exactly as I proposed. So people can use both URIs with a 
file: or just a plain path. URI.isAbsolute() returns false, if scheme ("file:") 
is missing: 
 "A 
URI is absolute if, and only if, it has a scheme component."


> Fix path usage for cloud backup/restore
> ---
>
> Key: SOLR-9444
> URL: https://issues.apache.org/jira/browse/SOLR-9444
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Varun Thacker
> Attachments: SOLR-9444.patch
>
>
> As noted by Uwe on 
> https://issues.apache.org/jira/browse/SOLR-9242?focusedCommentId=15438925=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15438925
>  the usage of URI#getPath is wrong. 
> Creating a Jira to track this better. More details to follow



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9451) Make clusterstatus command logging less verbose

[Varun Thacker (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Varun Thacker updated SOLR-9451:

Attachment: SOLR-9451.patch

Converted the logging lines to debug level 

> Make clusterstatus command logging less verbose
> ---
>
> Key: SOLR-9451
> URL: https://issues.apache.org/jira/browse/SOLR-9451
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Varun Thacker
>Assignee: Varun Thacker
>Priority: Minor
> Attachments: SOLR-9451.patch
>
>
> Today when we run a cluster status call, the logs get filled up with 
> ZkStateReader messages like this. If a user has a lot of collections this can 
> get quite verbose and likely not useful.
> {code}
> INFO  - 2016-08-29 12:10:55.533; [   ] 
> org.apache.solr.handler.admin.CollectionsHandler; Invoked Collection Action 
> :clusterstatus with params 
> json.nl=map=true=CLUSTERSTATUS=json and sendToOCPQueue=true
> INFO  - 2016-08-29 12:10:55.535; [   ] 
> org.apache.solr.common.cloud.ZkStateReader; Load collection config from: 
> [/collections/test1]
> INFO  - 2016-08-29 12:10:55.535; [   ] 
> org.apache.solr.common.cloud.ZkStateReader; path=[/collections/test1] 
> [configName]=[test1] specified config exists in ZooKeeper
> INFO  - 2016-08-29 12:10:55.536; [   ] 
> org.apache.solr.common.cloud.ZkStateReader; Load collection config from: 
> [/collections/gettingstarted]
> INFO  - 2016-08-29 12:10:55.537; [   ] 
> org.apache.solr.common.cloud.ZkStateReader; 
> path=[/collections/gettingstarted] [configName]=[gettingstarted] specified 
> config exists in ZooKeeper
> ..
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (SOLR-9451) Make clusterstatus command logging less verbose

[Varun Thacker (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Varun Thacker reassigned SOLR-9451:
---

Assignee: Varun Thacker

> Make clusterstatus command logging less verbose
> ---
>
> Key: SOLR-9451
> URL: https://issues.apache.org/jira/browse/SOLR-9451
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Varun Thacker
>Assignee: Varun Thacker
>Priority: Minor
>
> Today when we run a cluster status call, the logs get filled up with 
> ZkStateReader messages like this. If a user has a lot of collections this can 
> get quite verbose and likely not useful.
> {code}
> INFO  - 2016-08-29 12:10:55.533; [   ] 
> org.apache.solr.handler.admin.CollectionsHandler; Invoked Collection Action 
> :clusterstatus with params 
> json.nl=map=true=CLUSTERSTATUS=json and sendToOCPQueue=true
> INFO  - 2016-08-29 12:10:55.535; [   ] 
> org.apache.solr.common.cloud.ZkStateReader; Load collection config from: 
> [/collections/test1]
> INFO  - 2016-08-29 12:10:55.535; [   ] 
> org.apache.solr.common.cloud.ZkStateReader; path=[/collections/test1] 
> [configName]=[test1] specified config exists in ZooKeeper
> INFO  - 2016-08-29 12:10:55.536; [   ] 
> org.apache.solr.common.cloud.ZkStateReader; Load collection config from: 
> [/collections/gettingstarted]
> INFO  - 2016-08-29 12:10:55.537; [   ] 
> org.apache.solr.common.cloud.ZkStateReader; 
> path=[/collections/gettingstarted] [configName]=[gettingstarted] specified 
> config exists in ZooKeeper
> ..
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7430) Geo3d test failure

[Karl Wright (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15448165#comment-15448165
 ] 

Karl Wright commented on LUCENE-7430:
-

The north rectangle that is misbehaving is a narrow, very tiny rectangle with 
one point at the north pole and the other two points not far different from 
that.  The point is coming up as being not within the bounds but as being 
within the shape.  But this applies only to the quantized point; the 
unquantized point is fine.

So in this case it would initially appear that we have a test problem.

We generally accept that there are some effects of quantization; once a point 
is quantized it potentially leaves the exact surface of the world and can 
therefore wind up being outside of a shape's bounds.  That appears to be the 
situation here.  The only question is: what to do about it, without making the 
test useless?

> Geo3d test failure
> --
>
> Key: LUCENE-7430
> URL: https://issues.apache.org/jira/browse/LUCENE-7430
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Karl Wright
>Assignee: Karl Wright
>
> Here's the error msg:
> {code}
>[junit4] Suite: org.apache.lucene.spatial3d.TestGeo3DPoint
>[junit4] IGNOR/A 0.02s J0 | TestGeo3DPoint.testRandomBig
>[junit4]> Assumption #1: 'nightly' test group is disabled (@Nightly())
>[junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestGeo3DPoint 
> -Dtests.method=testRandomMedium -Dtests.seed=B167D2C466D257A0 
> -Dtests.slow=true -Dtests.locale=pt -Dtests.timezone=Etc/GMT+5 
> -Dtests.asserts=true -Dtests.file.encoding=Cp1252
>[junit4] FAILURE 4.06s J0 | TestGeo3DPoint.testRandomMedium <<<
>[junit4]> Throwable #1: java.lang.AssertionError: FAIL: id=4546 should 
> have matched but did not
>[junit4]>   shape=GeoWideNorthRectangle: 
> {planetmodel=PlanetModel.WGS84, 
> bottomlat=1.5707949696510948(89.2224138796), 
> leftlon=3.141592653589793(180.0), 
> rightlon=3.1140200112375265(178.42020396319145)}
>[junit4]>   bounds=XYZBounds: [xmin=-1.3551580601679786E-6 
> xmax=1.3551580601679786E-6 ymin=-1.3551580601679786E-6 
> ymax=1.3551580601679786E-6 zmin=0.9977622910211923 zmax=0.9977622930221051]
>[junit4]>   world bounds=( minX=-1.0011188539924791 
> maxX=1.0011188539924791 minY=-1.0011188539924791 maxY=1.0011188539924791 
> minZ=-0.9977622920221051 maxZ=0.9977622920221051
>[junit4]>   quantized point=[X=7.323492821176281E-6, 
> Y=-2.3309121299774915E-10, Z=0.9977622921205215] within shape? true within 
> bounds? false
>[junit4]>   unquantized point=[lat=1.570788986986606, 
> lon=-1.0056566715747591E-117([X=7.323383942914017E-6, 
> Y=-7.364809920694947E-123, Z=0.9977622919954089])] within shape? false within 
> bounds? false
>[junit4]>   docID=4438 deleted?=false
>[junit4]>   query=PointInGeo3DShapeQuery: field=point: Shape: 
> GeoWideNorthRectangle: {planetmodel=PlanetModel.WGS84, 
> bottomlat=1.5707949696510948(89.2224138796), 
> leftlon=3.141592653589793(180.0), 
> rightlon=3.1140200112375265(178.42020396319145)}
> ...
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org