[JENKINS] Lucene-Solr-7.x-Windows (64bit/jdk1.8.0_144) - Build # 384 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Windows/384/
Java: 64bit/jdk1.8.0_144 -XX:+UseCompressedOops -XX:+UseSerialGC

8 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.lucene.codecs.blockterms.TestVarGapDocFreqIntervalPostingsFormat

Error Message:
Could not remove the following files (in the order of attempts):
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\codecs\test\J0\temp\lucene.codecs.blockterms.TestVarGapDocFreqIntervalPostingsFormat_5A5F8707E8605BBC-001\testPostingsEnumReuse-001:
 java.nio.file.AccessDeniedException: 
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\codecs\test\J0\temp\lucene.codecs.blockterms.TestVarGapDocFreqIntervalPostingsFormat_5A5F8707E8605BBC-001\testPostingsEnumReuse-001

C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\codecs\test\J0\temp\lucene.codecs.blockterms.TestVarGapDocFreqIntervalPostingsFormat_5A5F8707E8605BBC-001:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\codecs\test\J0\temp\lucene.codecs.blockterms.TestVarGapDocFreqIntervalPostingsFormat_5A5F8707E8605BBC-001
 

Stack Trace:
java.io.IOException: Could not remove the following files (in the order of 
attempts):
   
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\codecs\test\J0\temp\lucene.codecs.blockterms.TestVarGapDocFreqIntervalPostingsFormat_5A5F8707E8605BBC-001\testPostingsEnumReuse-001:
 java.nio.file.AccessDeniedException: 
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\codecs\test\J0\temp\lucene.codecs.blockterms.TestVarGapDocFreqIntervalPostingsFormat_5A5F8707E8605BBC-001\testPostingsEnumReuse-001
   
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\codecs\test\J0\temp\lucene.codecs.blockterms.TestVarGapDocFreqIntervalPostingsFormat_5A5F8707E8605BBC-001:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\codecs\test\J0\temp\lucene.codecs.blockterms.TestVarGapDocFreqIntervalPostingsFormat_5A5F8707E8605BBC-001

at __randomizedtesting.SeedInfo.seed([5A5F8707E8605BBC]:0)
at org.apache.lucene.util.IOUtils.rm(IOUtils.java:329)
at 
org.apache.lucene.util.TestRuleTemporaryFilesCleanup.afterAlways(TestRuleTemporaryFilesCleanup.java:216)
at 
com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.afterAlways(TestRuleAdapter.java:31)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)


FAILED:  
org.apache.lucene.replicator.IndexAndTaxonomyReplicationClientTest.testConsistencyOnExceptions

Error Message:
Could not remove the following files (in the order of attempts):
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\replicator\test\J0\temp\lucene.replicator.IndexAndTaxonomyReplicationClientTest_8347783EA102BD6E-001\replicationClientTest-002\1:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\replicator\test\J0\temp\lucene.replicator.IndexAndTaxonomyReplicationClientTest_8347783EA102BD6E-001\replicationClientTest-002\1
 

Stack Trace:
java.io.IOException: Could not remove the following files (in the order of 
attempts):
   
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\replicator\test\J0\temp\lucene.replicator.IndexAndTaxonomyReplicationClientTest_8347783EA102BD6E-001\replicationClientTest-002\1:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\replicator\test\J0\temp\lucene.replicator.IndexAndTaxonomyReplicationClientTest_8347783EA102BD6E-001\replicationClientTest-002\1

at 
__randomizedtesting.SeedInfo.seed([8347783EA102BD6E:CC99F9EB36E4E91]:0)
at org.apache.lucene.util.IOUtils.rm(IOUtils.java:329)
at 
org.apache.lucene.replicator.PerSessionDirectoryFactory.cleanupSession(PerSessionDirectoryFactory.java:58)
at 
org.apache.lucene.replicator.ReplicationClient.doUpdate(ReplicationClient.java:259)
at 
org.apache.lucene.replicator.ReplicationClient.updateNow(ReplicationClient.java:401)
at 

[jira] [Commented] (SOLR-11741) Offline training mode for schema guessing

[Abhishek Kumar Singh (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16314428#comment-16314428
 ] 

Abhishek Kumar Singh commented on SOLR-11741:
-

What i was thinking was something similar to the above implementation, just 
that instead of recording every *value* that ever appeared for a field, I would 
record all the distinct *fieldTypes of the values* that appeared for a each 
field. This will be the mapping of *field -> supported types*. This will need 
very small storage.  

And instead of recording in memory, this data can be stored externally, (say 
_zookeeper_, or some _temporary index_ inside solr.). I think it will get rid 
of the following problem.

bq. It doesn't play very nicely with distributed updates (you'd either have to 
ensure all training data was sent to the same node where you send the "commit" 
or add special custom logic to ensure it all got forwarded to a special node) 
and there are probably a lot more sophisticated / smarter ways to do it



> Offline training mode for schema guessing
> -
>
> Key: SOLR-11741
> URL: https://issues.apache.org/jira/browse/SOLR-11741
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Ishan Chattopadhyaya
>
> Our data driven schema guessing doesn't work under many situations. For 
> example, if the first document has a field with value "0", it is guessed as 
> Long and subsequent fields with "0.0" are rejected. Similarly, if the same 
> field had alphanumeric contents for a latter document, those documents are 
> rejected. Also, single vs. multi valued field guessing is not ideal.
> Proposing an offline training mode where Solr accepts bunch of documents and 
> returns a guessed schema (without indexing). This schema can then be used for 
> actual indexing. I think the original idea is from Hoss.
> I think initial implementation can be based on an UpdateRequestProcessor. We 
> can hash out the API soon, as we go along.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-SmokeRelease-7.x - Build # 113 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-7.x/113/

No tests ran.

Build Log:
[...truncated 28289 lines...]
prepare-release-no-sign:
[mkdir] Created dir: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/build/smokeTestRelease/dist
 [copy] Copying 491 files to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/build/smokeTestRelease/dist/lucene
 [copy] Copying 215 files to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/build/smokeTestRelease/dist/solr
   [smoker] Java 1.8 JAVA_HOME=/home/jenkins/tools/java/latest1.8
   [smoker] NOTE: output encoding is UTF-8
   [smoker] 
   [smoker] Load release URL 
"file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/build/smokeTestRelease/dist/"...
   [smoker] 
   [smoker] Test Lucene...
   [smoker]   test basics...
   [smoker]   get KEYS
   [smoker] 0.2 MB in 0.05 sec (4.7 MB/sec)
   [smoker]   check changes HTML...
   [smoker]   download lucene-7.3.0-src.tgz...
   [smoker] 31.6 MB in 0.09 sec (359.6 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   download lucene-7.3.0.tgz...
   [smoker] 72.3 MB in 1.16 sec (62.5 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   download lucene-7.3.0.zip...
   [smoker] 82.8 MB in 2.49 sec (33.2 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   unpack lucene-7.3.0.tgz...
   [smoker] verify JAR metadata/identity/no javax.* or java.* classes...
   [smoker] test demo with 1.8...
   [smoker]   got 6282 hits for query "lucene"
   [smoker] checkindex with 1.8...
   [smoker] check Lucene's javadoc JAR
   [smoker]   unpack lucene-7.3.0.zip...
   [smoker] verify JAR metadata/identity/no javax.* or java.* classes...
   [smoker] test demo with 1.8...
   [smoker]   got 6282 hits for query "lucene"
   [smoker] checkindex with 1.8...
   [smoker] check Lucene's javadoc JAR
   [smoker]   unpack lucene-7.3.0-src.tgz...
   [smoker] make sure no JARs/WARs in src dist...
   [smoker] run "ant validate"
   [smoker] run tests w/ Java 8 and testArgs='-Dtests.slow=false'...
   [smoker] 
   [smoker] command "export JAVA_HOME="/home/jenkins/tools/java/latest1.8" 
PATH="/home/jenkins/tools/java/latest1.8/bin:$PATH" 
JAVACMD="/home/jenkins/tools/java/latest1.8/bin/java"; ant clean test 
-Dtests.slow=false" failed:
   [smoker] Buildfile: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/build/smokeTestRelease/tmp/unpack/lucene-7.3.0/build.xml
   [smoker] 
   [smoker] clean:
   [smoker][delete] Deleting directory 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/build/smokeTestRelease/tmp/unpack/lucene-7.3.0/build
   [smoker] 
   [smoker] ivy-availability-check:
   [smoker] [loadresource] Do not set property disallowed.ivy.jars.list as its 
length is 0.
   [smoker] 
   [smoker] -ivy-fail-disallowed-ivy-version:
   [smoker] 
   [smoker] ivy-fail:
   [smoker] 
   [smoker] ivy-configure:
   [smoker] [ivy:configure] :: Apache Ivy 2.4.0 - 20141213170938 :: 
http://ant.apache.org/ivy/ ::
   [smoker] [ivy:configure] :: loading settings :: file = 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/build/smokeTestRelease/tmp/unpack/lucene-7.3.0/top-level-ivy-settings.xml
   [smoker] 
   [smoker] -clover.load:
   [smoker] 
   [smoker] resolve-groovy:
   [smoker] [ivy:cachepath] :: resolving dependencies :: 
org.codehaus.groovy#groovy-all-caller;working
   [smoker] [ivy:cachepath] confs: [default]
   [smoker] [ivy:cachepath] found org.codehaus.groovy#groovy-all;2.4.12 in 
public
   [smoker] [ivy:cachepath] :: resolution report :: resolve 1115ms :: artifacts 
dl 16ms
   [smoker] 
-
   [smoker] |  |modules||   
artifacts   |
   [smoker] |   conf   | number| search|dwnlded|evicted|| 
number|dwnlded|
   [smoker] 
-
   [smoker] |  default |   1   |   0   |   0   |   0   ||   1   |   
0   |
   [smoker] 
-
   [smoker] 
   [smoker] -init-totals:
   [smoker] 
   [smoker] test-core:
   [smoker] 
   [smoker] -clover.disable:
   [smoker] 
   [smoker] ivy-availability-check:
   [smoker] [loadresource] Do not set property disallowed.ivy.jars.list as its 
length is 0.
   [smoker] 
   [smoker] -ivy-fail-disallowed-ivy-version:
   [smoker] 
   [smoker] ivy-fail:
   [smoker] 
   [smoker] ivy-configure:
   [smoker] [ivy:configure] :: loading settings :: file = 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/build/smokeTestRelease/tmp/unpack/lucene-7.3.0/top-level-ivy-settings.xml
   [smoker] 
   [smoker] -clover.load:
   [smoker] 
   [smoker] -clover.classpath:
   [smoker] 
   [smoker] 

Re: BugFix release 7.2.1

[S G]

Just curious to know, does the test suite include some performance test
also?
I would like to know the performance impact of using pints vs tints or ints
etc.
If they are not there, I can try to add some tests for the same.

Thanks
SG


On Fri, Jan 5, 2018 at 5:47 PM, Đạt Cao Mạnh 
wrote:

> Hi all,
>
> I will work on SOLR-11771
>  today, It is a simple
> fix and will be great if it get fixed in 7.2.1
>
> On Fri, Jan 5, 2018 at 11:23 PM Erick Erickson 
> wrote:
>
>> Neither of those Solr fixes are earth shatteringly important, they've
>> both been around for quite a while. I don't think it's urgent to include
>> them.
>>
>> That said, they're pretty simple and isolated so worth doing if Jim is
>> willing. But not worth straining much. I was just clearing out some backlog
>> over vacation.
>>
>> Strictly up to you Jim.
>>
>> Erick
>>
>> On Fri, Jan 5, 2018 at 6:54 AM, David Smiley 
>> wrote:
>>
>>> https://issues.apache.org/jira/browse/SOLR-11809 is in progress, should
>>> be easy and I think definitely worth backporting
>>>
>>> On Fri, Jan 5, 2018 at 8:52 AM Adrien Grand  wrote:
>>>
 +1

 Looking at the changelog, 7.3 has 3 bug fixes for now: LUCENE-8077,
 SOLR-11783 and SOLR-11555. The Lucene change doesn't seem worth
 backporting, but maybe the Solr changes should?

 Le ven. 5 janv. 2018 à 12:40, jim ferenczi  a
 écrit :

> Hi,
> We discovered a bad bug in 7x that affects indices created in 6x with
> Lucene54DocValues format. The SortedNumericDocValues created with this
> format have a bug when advanceExact is used, the values retrieved for the
> docs when advanceExact returns true are invalid (the pointer to the values
> is not updated):
> https://issues.apache.org/jira/browse/LUCENE-8117
> This affects all indices created in 6x with sorted numeric doc values
> so I wanted to ask if anyone objects to a bugfix release for 7.2 (7.2.1). 
> I
> also volunteer to be the release manager for this one if it is accepted.
>
> Jim
>

>>>
>>> --
>>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>>> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.
>>> solrenterprisesearchserver.com
>>>
>>
>>

[JENKINS] Lucene-Solr-master-Windows (64bit/jdk-9.0.1) - Build # 7096 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/7096/
Java: 64bit/jdk-9.0.1 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

9 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.lucene.search.TestShardSearching

Error Message:
Could not remove the following files (in the order of attempts):
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\core\test\J1\temp\lucene.search.TestShardSearching_B54F45F753F65BA7-001\ShardSearchingTestBase-003:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\core\test\J1\temp\lucene.search.TestShardSearching_B54F45F753F65BA7-001\ShardSearchingTestBase-003

C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\core\test\J1\temp\lucene.search.TestShardSearching_B54F45F753F65BA7-001\ShardSearchingTestBase-003\_0.cfs:
 java.nio.file.AccessDeniedException: 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\core\test\J1\temp\lucene.search.TestShardSearching_B54F45F753F65BA7-001\ShardSearchingTestBase-003\_0.cfs

C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\core\test\J1\temp\lucene.search.TestShardSearching_B54F45F753F65BA7-001:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\core\test\J1\temp\lucene.search.TestShardSearching_B54F45F753F65BA7-001
 

Stack Trace:
java.io.IOException: Could not remove the following files (in the order of 
attempts):
   
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\core\test\J1\temp\lucene.search.TestShardSearching_B54F45F753F65BA7-001\ShardSearchingTestBase-003:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\core\test\J1\temp\lucene.search.TestShardSearching_B54F45F753F65BA7-001\ShardSearchingTestBase-003
   
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\core\test\J1\temp\lucene.search.TestShardSearching_B54F45F753F65BA7-001\ShardSearchingTestBase-003\_0.cfs:
 java.nio.file.AccessDeniedException: 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\core\test\J1\temp\lucene.search.TestShardSearching_B54F45F753F65BA7-001\ShardSearchingTestBase-003\_0.cfs
   
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\core\test\J1\temp\lucene.search.TestShardSearching_B54F45F753F65BA7-001:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\core\test\J1\temp\lucene.search.TestShardSearching_B54F45F753F65BA7-001

at __randomizedtesting.SeedInfo.seed([B54F45F753F65BA7]:0)
at org.apache.lucene.util.IOUtils.rm(IOUtils.java:329)
at 
org.apache.lucene.util.TestRuleTemporaryFilesCleanup.afterAlways(TestRuleTemporaryFilesCleanup.java:216)
at 
com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.afterAlways(TestRuleAdapter.java:31)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  
org.apache.lucene.replicator.IndexAndTaxonomyReplicationClientTest.testRestart

Error Message:
Could not remove the following files (in the order of attempts):
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\replicator\test\J1\temp\lucene.replicator.IndexAndTaxonomyReplicationClientTest_5FDD548F3A860EF2-001\replicationClientTest-001\3\index:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\replicator\test\J1\temp\lucene.replicator.IndexAndTaxonomyReplicationClientTest_5FDD548F3A860EF2-001\replicationClientTest-001\3\index

C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\replicator\test\J1\temp\lucene.replicator.IndexAndTaxonomyReplicationClientTest_5FDD548F3A860EF2-001\replicationClientTest-001\3:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\replicator\test\J1\temp\lucene.replicator.IndexAndTaxonomyReplicationClientTest_5FDD548F3A860EF2-001\replicationClientTest-001\3
 

Stack Trace:
java.io.IOException: Could 

[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 115 - Still unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/115/

16 tests failed.
FAILED:  
org.apache.solr.cloud.TestCloudRecovery.leaderRecoverFromLogOnStartupTest

Error Message:
Timeout waiting for all live and active

Stack Trace:
java.lang.AssertionError: Timeout waiting for all live and active
at 
__randomizedtesting.SeedInfo.seed([E4D1137015757DB0:9021F23551513A3F]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.cloud.TestCloudRecovery.leaderRecoverFromLogOnStartupTest(TestCloudRecovery.java:99)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.TestCloudRecovery.corruptedLogTest

Error Message:
Could not find a healthy node to handle the request.

Stack Trace:
org.apache.solr.common.SolrException: Could not find a healthy node to handle 
the request.
at 

[jira] [Updated] (SOLR-11758) Missing boolVal implementation in FloatDocValues

[Hoss Man (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hoss Man updated SOLR-11758:

Attachment: SOLR-11758.patch

Nice catch!

I expanded on your test a bit to prove that all of the numeric DocValue 
abstract FunctionValue classes follow the same expected semantics for 
{{boolVal()}} and added some sanity check tests at the solr level of the 
specific type of failure you encountered.

I'll commit soon unless anyone else spots any problems.

> Missing boolVal implementation in FloatDocValues
> 
>
> Key: SOLR-11758
> URL: https://issues.apache.org/jira/browse/SOLR-11758
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Munendra S N
>Priority: Minor
> Attachments: SOLR-11758.patch, SOLR-11758.patch
>
>   Original Estimate: 2h
>  Remaining Estimate: 2h
>
> In case of *DoubleDocValues*, *boolVal(int doc)* has been implemented but 
> this is missing in case *FloatDocValues*.
> *Impact*: For any DocValues which extends *FloatDocValues* and doesn't 
> implement *boolVal(int doc)*, parent *boolVal* in FucntionValues would be 
> called.
> boolVal implementation in FunctionValues
> {code:java}
>   @Override
>   public boolean boolVal(int doc) throws IOException {
> return intVal(doc) != 0;
>   }
> {code}
> Let me know if I can work on it



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (SOLR-11758) Missing boolVal implementation in FloatDocValues

[Hoss Man (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hoss Man reassigned SOLR-11758:
---

Assignee: Hoss Man

> Missing boolVal implementation in FloatDocValues
> 
>
> Key: SOLR-11758
> URL: https://issues.apache.org/jira/browse/SOLR-11758
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Munendra S N
>Assignee: Hoss Man
>Priority: Minor
> Attachments: SOLR-11758.patch, SOLR-11758.patch
>
>   Original Estimate: 2h
>  Remaining Estimate: 2h
>
> In case of *DoubleDocValues*, *boolVal(int doc)* has been implemented but 
> this is missing in case *FloatDocValues*.
> *Impact*: For any DocValues which extends *FloatDocValues* and doesn't 
> implement *boolVal(int doc)*, parent *boolVal* in FucntionValues would be 
> called.
> boolVal implementation in FunctionValues
> {code:java}
>   @Override
>   public boolean boolVal(int doc) throws IOException {
> return intVal(doc) != 0;
>   }
> {code}
> Let me know if I can work on it



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-7.x-Solaris (64bit/jdk1.8.0) - Build # 376 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Solaris/376/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseSerialGC

2 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation

Error Message:
3 threads leaked from SUITE scope at 
org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation: 1) 
Thread[id=11305, name=jetty-launcher-4197-thread-2-SendThread(127.0.0.1:41494), 
state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] 
at java.lang.Thread.sleep(Native Method) at 
org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
 at 
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)
 at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)
2) Thread[id=11306, name=jetty-launcher-4197-thread-2-EventThread, 
state=WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] at 
sun.misc.Unsafe.park(Native Method) at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
 at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) 
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)
3) Thread[id=11235, name=jetty-launcher-4197-thread-1-EventThread, 
state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] 
at sun.misc.Unsafe.park(Native Method) at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) 
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
 at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
 at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)  
   at 
org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
 at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)  
   at 
org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
 at 
org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
 at 
org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
 at 
org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
 at 
org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
 at 
org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
 at 
org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
 at 
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)   
  at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 3 threads leaked from SUITE 
scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation: 
   1) Thread[id=11305, 
name=jetty-launcher-4197-thread-2-SendThread(127.0.0.1:41494), 
state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at java.lang.Thread.sleep(Native Method)
at 
org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
at 
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)
   2) Thread[id=11306, name=jetty-launcher-4197-thread-2-EventThread, 
state=WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)
   3) Thread[id=11235, name=jetty-launcher-4197-thread-1-EventThread, 
state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at sun.misc.Unsafe.park(Native Method)
at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
at 
org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
  

[JENKINS] Lucene-Solr-master-Linux (64bit/jdk1.8.0_144) - Build # 21222 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/21222/
Java: 64bit/jdk1.8.0_144 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC

9 tests failed.
FAILED:  
org.apache.solr.cloud.CollectionsAPISolrJTest.testCreateWithDefaultConfigSet

Error Message:
Error from server at https://127.0.0.1:34289/solr: Could not fully remove 
collection: solrj_default_configset

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at https://127.0.0.1:34289/solr: Could not fully remove collection: 
solrj_default_configset
at 
__randomizedtesting.SeedInfo.seed([3936C1E021EF3E5E:779D60A77E5044AD]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at 
org.apache.solr.cloud.CollectionsAPISolrJTest.testCreateWithDefaultConfigSet(CollectionsAPISolrJTest.java:81)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[jira] [Created] (SOLR-11827) MockAuthorizationPlugin should return 401 if no principal is specified

[Varun Thacker (JIRA)]

Varun Thacker created SOLR-11827:


 Summary: MockAuthorizationPlugin should return 401 if no principal 
is specified
 Key: SOLR-11827
 URL: https://issues.apache.org/jira/browse/SOLR-11827
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Varun Thacker


Let's say today if the leader sends a message to the replica and it takes more 
than 10s ( the default TTL timeout ) then PKIAuthenticationPlugin will not pass 
the principal and RuleBasedAuthorizationPlugin will notice this and throw a 401

{code:title=PKIAuthenticationPlugin.java|borderStyle=solid}
if ((receivedTime - decipher.timestamp) > MAX_VALIDITY) {
log.error("Invalid key request timestamp: {} , received timestamp: {} , 
TTL: {}", decipher.timestamp, receivedTime, MAX_VALIDITY);
filterChain.doFilter(request, response);
return true;
}
{code}

{code:title=RuleBasedAuthorizationPlugin.java|borderStyle=solid}
if (principal == null) {
log.info("request has come without principal. failed permission {} 
",permission);
//this resource needs a principal but the request has come without
//any credential.
return MatchStatus.USER_REQUIRED;
  }
{code}

I was trying to verify this with PKIAuthenticationIntegrationTest but I noticed 
that since this test uses MockAuthorizationPlugin where no principal is treated 
as a 200 the test won't fail.

So we should enhance MockAuthorizationPlugin to treat no principal as a 401 and 
add a test in PKIAuthenticationIntegrationTest to verify the behaviour




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-11826) TestPKIAuthenticationPlugin NPE

[Varun Thacker (JIRA)]

Varun Thacker created SOLR-11826:


 Summary: TestPKIAuthenticationPlugin NPE
 Key: SOLR-11826
 URL: https://issues.apache.org/jira/browse/SOLR-11826
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Varun Thacker
 Attachments: jenkins_build19_72.log

I see Jenkins failures for TestPKIAuthenticationPlugin.

SOLR-9401 we addressed a few scenarios but maybe there are some more cases 
where we run into this.

I'll upload logs from one such scenario for reference, but the test fails quite 
regularly . 

This link is still active 
https://builds.apache.org/job/Lucene-Solr-Tests-7.2/19/ and I've uploaded the 
logs from this test scenario for reference.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-11771) Overseer can never process some last messages

[Cao Manh Dat (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Cao Manh Dat resolved SOLR-11771.
-
   Resolution: Fixed
 Assignee: Cao Manh Dat
Fix Version/s: 7.3
   master (8.0)

> Overseer can never process some last messages
> -
>
> Key: SOLR-11771
> URL: https://issues.apache.org/jira/browse/SOLR-11771
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Cao Manh Dat
>Assignee: Cao Manh Dat
> Fix For: master (8.0), 7.3
>
>
> I met this problem when some tests get failed because timeout on creating a 
> new collection. The case here is
> 1. Overseer call DQ.peekElements with LONG.MAX_VALUE timeout.
> 2. DQ fetch children from ZK but it is empty.
> 3. At the same time with 2, a set LEADER message is enqueued, 
> {{changed.signalAll()}} already called.
> 4. The call {{changed.awaitNanos(waitNanos)}} is never end because no new 
> messages get enqueued after this point.
> The fix here is trivial, the call peekElements should have a timeout of 2 
> seconds, that will be appropriate in most cases.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11771) Overseer can never process some last messages

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16314302#comment-16314302
 ] 

ASF subversion and git services commented on SOLR-11771:


Commit 169850f90d0db65b3b9490384a8e542e478960e5 in lucene-solr's branch 
refs/heads/branch_7x from [~caomanhdat]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=169850f ]

SOLR-11771: Move CHANGES entry to the right version


> Overseer can never process some last messages
> -
>
> Key: SOLR-11771
> URL: https://issues.apache.org/jira/browse/SOLR-11771
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Cao Manh Dat
>
> I met this problem when some tests get failed because timeout on creating a 
> new collection. The case here is
> 1. Overseer call DQ.peekElements with LONG.MAX_VALUE timeout.
> 2. DQ fetch children from ZK but it is empty.
> 3. At the same time with 2, a set LEADER message is enqueued, 
> {{changed.signalAll()}} already called.
> 4. The call {{changed.awaitNanos(waitNanos)}} is never end because no new 
> messages get enqueued after this point.
> The fix here is trivial, the call peekElements should have a timeout of 2 
> seconds, that will be appropriate in most cases.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11771) Overseer can never process some last messages

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16314301#comment-16314301
 ] 

ASF subversion and git services commented on SOLR-11771:


Commit 8eed452ce7a5030bdcde5d283ca2f35e4546d478 in lucene-solr's branch 
refs/heads/branch_7x from [~caomanhdat]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=8eed452 ]

SOLR-11771: Overseer can never process some last messages


> Overseer can never process some last messages
> -
>
> Key: SOLR-11771
> URL: https://issues.apache.org/jira/browse/SOLR-11771
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Cao Manh Dat
>
> I met this problem when some tests get failed because timeout on creating a 
> new collection. The case here is
> 1. Overseer call DQ.peekElements with LONG.MAX_VALUE timeout.
> 2. DQ fetch children from ZK but it is empty.
> 3. At the same time with 2, a set LEADER message is enqueued, 
> {{changed.signalAll()}} already called.
> 4. The call {{changed.awaitNanos(waitNanos)}} is never end because no new 
> messages get enqueued after this point.
> The fix here is trivial, the call peekElements should have a timeout of 2 
> seconds, that will be appropriate in most cases.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11771) Overseer can never process some last messages

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16314300#comment-16314300
 ] 

ASF subversion and git services commented on SOLR-11771:


Commit dd2c90d7680a54d3eefb43b26cde4c2ede265928 in lucene-solr's branch 
refs/heads/branch_7_2 from [~caomanhdat]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=dd2c90d ]

SOLR-11771: Overseer can never process some last messages


> Overseer can never process some last messages
> -
>
> Key: SOLR-11771
> URL: https://issues.apache.org/jira/browse/SOLR-11771
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Cao Manh Dat
>
> I met this problem when some tests get failed because timeout on creating a 
> new collection. The case here is
> 1. Overseer call DQ.peekElements with LONG.MAX_VALUE timeout.
> 2. DQ fetch children from ZK but it is empty.
> 3. At the same time with 2, a set LEADER message is enqueued, 
> {{changed.signalAll()}} already called.
> 4. The call {{changed.awaitNanos(waitNanos)}} is never end because no new 
> messages get enqueued after this point.
> The fix here is trivial, the call peekElements should have a timeout of 2 
> seconds, that will be appropriate in most cases.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11771) Overseer can never process some last messages

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16314297#comment-16314297
 ] 

ASF subversion and git services commented on SOLR-11771:


Commit 0d5581177423aef95489351e07b2169150a9c102 in lucene-solr's branch 
refs/heads/master from [~caomanhdat]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=0d55811 ]

SOLR-11771: Move CHANGES entry to the right version


> Overseer can never process some last messages
> -
>
> Key: SOLR-11771
> URL: https://issues.apache.org/jira/browse/SOLR-11771
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Cao Manh Dat
>
> I met this problem when some tests get failed because timeout on creating a 
> new collection. The case here is
> 1. Overseer call DQ.peekElements with LONG.MAX_VALUE timeout.
> 2. DQ fetch children from ZK but it is empty.
> 3. At the same time with 2, a set LEADER message is enqueued, 
> {{changed.signalAll()}} already called.
> 4. The call {{changed.awaitNanos(waitNanos)}} is never end because no new 
> messages get enqueued after this point.
> The fix here is trivial, the call peekElements should have a timeout of 2 
> seconds, that will be appropriate in most cases.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11771) Overseer can never process some last messages

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16314296#comment-16314296
 ] 

ASF subversion and git services commented on SOLR-11771:


Commit b8b00786f377af30e3572282e89dbbd32fbef742 in lucene-solr's branch 
refs/heads/master from [~caomanhdat]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b8b00786 ]

SOLR-11771: Overseer can never process some last messages


> Overseer can never process some last messages
> -
>
> Key: SOLR-11771
> URL: https://issues.apache.org/jira/browse/SOLR-11771
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Cao Manh Dat
>
> I met this problem when some tests get failed because timeout on creating a 
> new collection. The case here is
> 1. Overseer call DQ.peekElements with LONG.MAX_VALUE timeout.
> 2. DQ fetch children from ZK but it is empty.
> 3. At the same time with 2, a set LEADER message is enqueued, 
> {{changed.signalAll()}} already called.
> 4. The call {{changed.awaitNanos(waitNanos)}} is never end because no new 
> messages get enqueued after this point.
> The fix here is trivial, the call peekElements should have a timeout of 2 
> seconds, that will be appropriate in most cases.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11771) Overseer can never process some last messages

[Cao Manh Dat (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Cao Manh Dat updated SOLR-11771:

Summary: Overseer can never process some last messages  (was: Overseer can 
never process some last messages due to a bug in DQ.peekElements())

> Overseer can never process some last messages
> -
>
> Key: SOLR-11771
> URL: https://issues.apache.org/jira/browse/SOLR-11771
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Cao Manh Dat
>
> I met this problem when some tests get failed because timeout on creating a 
> new collection. The case here is
> 1. Overseer call DQ.peekElements with LONG.MAX_VALUE timeout.
> 2. DQ fetch children from ZK but it is empty.
> 3. At the same time with 2, a set LEADER message is enqueued, 
> {{changed.signalAll()}} already called.
> 4. The call {{changed.awaitNanos(waitNanos)}} is never end because no new 
> messages get enqueued after this point.
> The fix here is trivial, the call peekElements should have a timeout of 2 
> seconds, that will be appropriate in most cases.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11771) Overseer can never process some last messages due to a bug in DQ.peekElements()

[Cao Manh Dat (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Cao Manh Dat updated SOLR-11771:

Summary: Overseer can never process some last messages due to a bug in 
DQ.peekElements()  (was: Overseer can never process some last messages)

> Overseer can never process some last messages due to a bug in 
> DQ.peekElements()
> ---
>
> Key: SOLR-11771
> URL: https://issues.apache.org/jira/browse/SOLR-11771
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Cao Manh Dat
>
> I met this problem when some tests get failed because timeout on creating a 
> new collection. The case here is
> 1. Overseer call DQ.peekElements with LONG.MAX_VALUE timeout.
> 2. DQ fetch children from ZK but it is empty.
> 3. At the same time with 2, a set LEADER message is enqueued, 
> {{changed.signalAll()}} already called.
> 4. The call {{changed.awaitNanos(waitNanos)}} is never end because no new 
> messages get enqueued after this point.
> The fix here is trivial, the call peekElements should have a timeout of 2 
> seconds, that will be appropriate in most cases.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: BugFix release 7.2.1

[Đạt Cao Mạnh]

Hi all,

I will work on SOLR-11771
 today,
It is a simple fix and will be great if it get fixed in 7.2.1

On Fri, Jan 5, 2018 at 11:23 PM Erick Erickson 
wrote:

> Neither of those Solr fixes are earth shatteringly important, they've both
> been around for quite a while. I don't think it's urgent to include them.
>
> That said, they're pretty simple and isolated so worth doing if Jim is
> willing. But not worth straining much. I was just clearing out some backlog
> over vacation.
>
> Strictly up to you Jim.
>
> Erick
>
> On Fri, Jan 5, 2018 at 6:54 AM, David Smiley 
> wrote:
>
>> https://issues.apache.org/jira/browse/SOLR-11809 is in progress, should
>> be easy and I think definitely worth backporting
>>
>> On Fri, Jan 5, 2018 at 8:52 AM Adrien Grand  wrote:
>>
>>> +1
>>>
>>> Looking at the changelog, 7.3 has 3 bug fixes for now: LUCENE-8077,
>>> SOLR-11783 and SOLR-11555. The Lucene change doesn't seem worth
>>> backporting, but maybe the Solr changes should?
>>>
>>> Le ven. 5 janv. 2018 à 12:40, jim ferenczi  a
>>> écrit :
>>>
 Hi,
 We discovered a bad bug in 7x that affects indices created in 6x with
 Lucene54DocValues format. The SortedNumericDocValues created with this
 format have a bug when advanceExact is used, the values retrieved for the
 docs when advanceExact returns true are invalid (the pointer to the values
 is not updated):
 https://issues.apache.org/jira/browse/LUCENE-8117
 This affects all indices created in 6x with sorted numeric doc values
 so I wanted to ask if anyone objects to a bugfix release for 7.2 (7.2.1). I
 also volunteer to be the release manager for this one if it is accepted.

 Jim

>>>
>>
>> --
>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>> LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
>> http://www.solrenterprisesearchserver.com
>>
>
>

[JENKINS] Lucene-Solr-7.x-Linux (64bit/jdk1.8.0_144) - Build # 1127 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/1127/
Java: 64bit/jdk1.8.0_144 -XX:+UseCompressedOops -XX:+UseG1GC

3 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation

Error Message:
2 threads leaked from SUITE scope at 
org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation: 1) 
Thread[id=18937, name=jetty-launcher-3303-thread-2-EventThread, 
state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] 
at sun.misc.Unsafe.park(Native Method) at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) 
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
 at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
 at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)  
   at 
org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
 at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)  
   at 
org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
 at 
org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
 at 
org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
 at 
org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
 at 
org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
 at 
org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
 at 
org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
 at 
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)   
  at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)   
 2) Thread[id=18933, name=jetty-launcher-3303-thread-1-EventThread, 
state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] 
at sun.misc.Unsafe.park(Native Method) at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) 
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
 at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
 at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)  
   at 
org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
 at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)  
   at 
org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
 at 
org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
 at 
org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
 at 
org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
 at 
org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
 at 
org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
 at 
org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
 at 
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)   
  at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE 
scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation: 
   1) Thread[id=18937, name=jetty-launcher-3303-thread-2-EventThread, 
state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at sun.misc.Unsafe.park(Native Method)
at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
at 
org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
at 
org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
at 
org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
at 

[jira] [Commented] (SOLR-11739) Solr can accept duplicated async IDs

[Anshum Gupta (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16314192#comment-16314192
 ] 

Anshum Gupta commented on SOLR-11739:
-

Thanks for taking this up [~tomasflobbe]! This looks good and not super 
invasive. 

Here's some feedback:
* CollectionsHandler.java
** We should add back the check in old places instead of leaving that as a TODO
** L296: It would be good to log the reason for the failure to offer the 
request to the OverseerCollectionQueue
* Are the log4j.properties changes intentional ?
* SizeLimitedDistributedMap.OnChildDeleteObserver interface doesn’t need the 
keyword static also can you add some javadoc there?
* As I understand, you would also want to override clear() and call 
onDeleteObserver.onChildDelete in those cases too.
* Javadocs for claim/clearAsyncId. It’d be good to mention that it returns 
false when it tries to clear an ID that doesn’t exist any more.

> Solr can accept duplicated async IDs
> 
>
> Key: SOLR-11739
> URL: https://issues.apache.org/jira/browse/SOLR-11739
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Tomás Fernández Löbbe
>Assignee: Tomás Fernández Löbbe
>Priority: Minor
> Attachments: SOLR-11739.patch, SOLR-11739.patch
>
>
> Solr is supposed to reject duplicated async IDs, however, if the repeated IDs 
> are sent fast enough, a race condition in Solr will let the repeated IDs 
> through. The duplicated task is ran and and then silently fails to report as 
> completed because the same async ID is already in the completed map. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-7.x-MacOSX (64bit/jdk-9) - Build # 382 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-MacOSX/382/
Java: 64bit/jdk-9 -XX:+UseCompressedOops -XX:+UseG1GC

1 tests failed.
FAILED:  org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testBasic

Error Message:


Stack Trace:
java.util.ConcurrentModificationException
at 
__randomizedtesting.SeedInfo.seed([A48A5F020D2EBCDB:F704217D2F23AF5]:0)
at 
java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:937)
at java.base/java.util.ArrayList$Itr.next(ArrayList.java:891)
at 
org.apache.solr.cloud.autoscaling.sim.SimSolrCloudTestCase.tearDown(SimSolrCloudTestCase.java:141)
at jdk.internal.reflect.GeneratedMethodAccessor84.invoke(Unknown Source)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)




Build Log:
[...truncated 13583 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster
   [junit4]   2> 3309070 INFO  
(SUITE-TestLargeCluster-seed#[A48A5F020D2EBCDB]-worker) [] 
o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: 
test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: 
/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.sim.TestLargeCluster_A48A5F020D2EBCDB-001/init-core-data-001
   [junit4]   2> 3309071 INFO  
(SUITE-TestLargeCluster-seed#[A48A5F020D2EBCDB]-worker) [   

[jira] [Commented] (SOLR-11741) Offline training mode for schema guessing

[Hoss Man (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16314169#comment-16314169
 ] 

Hoss Man commented on SOLR-11741:
-

bq. Proposing an offline training mode where Solr accepts bunch of documents 
and returns a guessed schema (without indexing). This schema can then be used 
for actual indexing. I think the original idea is from Hoss.

bq. I think initial implementation can be based on an UpdateRequestProcessor. 
We can hash out the API soon, as we go along.

FWIW...

What i suggested at one point (I don't remember where ... it may already be in 
a jira somewhere?) was an UpdateRequestProcessorFactory that could be 
configured _instead_ of RunUpdateProcessorFactory in a chain (that does not 
already include AddSchemaFieldsUpdateProcessorFactory) *after* all of the 
various ParseFooFieldUpdateProcessorFactories.

* For ADD commands, this processor (factory) would iterate over the 
SolrInputDocuments, and then iterate over the field names in those documents, 
and record in memory wether any docs had more then one value for that field 
name, as well as the "Least Common Denominator" of the *java type* of the 
values found -- ie: 
** if docA=1(int), but docB=1.1(float), docC=5.5(float) then we remember "Float"
** if docA=1(int), docB=1.1(float), docC=1.01(double) then we remember 
"Double".
** If docA=2017-12-12Z(date) and docB=42(int) then we remember "String"
* For COMMIT commands, this processor (factory) would take all of the info it 
had accumulated from the ADDs recieved up to that point, and use them to exec 
Schema Field additions -- using the same sort of "java object class -> 
fieldType name" mapping that AddSchemaFieldsUpdateProcessorFactory

The idea being that instead of having full "schemaless" mode enabled, there 
could be an {{/update/train-schema}} RequestHanlder configured to use this 
{{update.chain}}  Users could post a sampling of their docs to 
{{/update/train-schema}} then once they were don training send a 
{{/update/train-schema?commit=true}} command and the processor (factory) would 
add all the needed fields.


By no means should that idea be considered an end all be all solution / design.

It doesn't play very nicely with distributed updates (you'd either have to 
ensure all training data was sent to the same node where you send the "commit" 
or add special custom logic to ensure it all got forwarded to a special node) 
and there are probably a lot more sophisticated / smarter ways to do it ... it 
was just something i brainstormed one day as something that should be fairly 
easy to implement as a solr plugin leveraging most of the existing "schemaless" 
features of Solr -- where "Parse if possible" update processors already do most 
of the heavy lifting.  

Perhaps it can inspire a more robust solution?

> Offline training mode for schema guessing
> -
>
> Key: SOLR-11741
> URL: https://issues.apache.org/jira/browse/SOLR-11741
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Ishan Chattopadhyaya
>
> Our data driven schema guessing doesn't work under many situations. For 
> example, if the first document has a field with value "0", it is guessed as 
> Long and subsequent fields with "0.0" are rejected. Similarly, if the same 
> field had alphanumeric contents for a latter document, those documents are 
> rejected. Also, single vs. multi valued field guessing is not ideal.
> Proposing an offline training mode where Solr accepts bunch of documents and 
> returns a guessed schema (without indexing). This schema can then be used for 
> actual indexing. I think the original idea is from Hoss.
> I think initial implementation can be based on an UpdateRequestProcessor. We 
> can hash out the API soon, as we go along.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11825) TestPointFields.testDatePointFieldSortAndFunction() failure

[Hoss Man (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16314093#comment-16314093
 ] 

Hoss Man commented on SOLR-11825:
-

yeah -- i was just about to post that hypothosis based on reading the query 
response and noting how close those "randomly large" dates were to eachother.

I suspect the test, as written to (evidently) use completley random dates, is 
invalid given that sorting by function happens at float precision.

We should probably make this test only used randomized dates for strict date 
sorting (ie: {{sort=field asc|desc}} ) since that should happen at ms level 
precision -- and should fail if it doesn't -- and use a static set of dates for 
testing that function composition/sorting works properly

> TestPointFields.testDatePointFieldSortAndFunction() failure
> ---
>
> Key: SOLR-11825
> URL: https://issues.apache.org/jira/browse/SOLR-11825
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Steve Rowe
>
> Reproducing master seed from 
> [https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/21216/]:
> {noformat}
> Checking out Revision 5a08fa8bbb1cf26b4af5b71549671c31e1427f44 
> (refs/remotes/origin/master)
> [...]
>[junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestPointFields 
> -Dtests.method=testDatePointFieldSortAndFunction 
> -Dtests.seed=A41248828EFF34E3 -Dtests.multiplier=3 -Dtests.slow=true 
> -Dtests.locale=ar-YE -Dtests.timezone=Etc/GMT+10 -Dtests.asserts=true 
> -Dtests.file.encoding=UTF-8
>[junit4] ERROR   0.17s J2 | 
> TestPointFields.testDatePointFieldSortAndFunction <<<
>[junit4]> Throwable #1: java.lang.RuntimeException: Exception during 
> query
>[junit4]>  at 
> __randomizedtesting.SeedInfo.seed([A41248828EFF34E3:73AF779D2D1E9BFD]:0)
>[junit4]>  at 
> org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:902)
>[junit4]>  at 
> org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:862)
>[junit4]>  at 
> org.apache.solr.schema.TestPointFields.doTestDatePointFunctionQuery(TestPointFields.java:3599)
>[junit4]>  at 
> org.apache.solr.schema.TestPointFields.testDatePointFieldSortAndFunction(TestPointFields.java:1664)
>[junit4]>  at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>[junit4]>  at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>[junit4]>  at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>[junit4]>  at 
> java.base/java.lang.reflect.Method.invoke(Method.java:564)
>[junit4]>  at java.base/java.lang.Thread.run(Thread.java:844)
>[junit4]> Caused by: java.lang.RuntimeException: REQUEST FAILED: 
> xpath=//result/doc[9]/date[@name='number_p_dt_dv'][.='+293401-11-02T19:17:28.572Z']
>[junit4]>  xml response was: 
>[junit4]> 
>[junit4]> 0 name="QTime">0 start="0">R name="number_p_dt_dv">+831895-07-23T21:09:09.338Z name="id">Z name="number_p_dt_dv">+725281-03-30T13:09:50.329Z name="id">A name="number_p_dt_dv">+681046-11-04T12:49:38.424Z name="id">S name="number_p_dt_dv">+662906-09-27T18:36:37.903Z name="id">K name="number_p_dt_dv">+477986-01-24T14:48:55.265Z name="id">M name="number_p_dt_dv">+395512-11-01T10:23:52.727Z name="id">I name="number_p_dt_dv">+350980-09-17T07:18:22.252Z name="id">O name="number_p_dt_dv">+295226-09-03T14:25:02.014Z name="id">F name="number_p_dt_dv">+293401-10-30T14:59:03.236Z name="id">Y name="number_p_dt_dv">+293401-11-02T19:17:28.572Z name="id">N name="number_p_dt_dv">+284573-12-19T12:15:27.792Z name="id">X name="number_p_dt_dv">+223248-07-13T00:56:07.425Z name="id">U name="number_p_dt_dv">+13953-10-02T03:24:10.232Z name="id">E name="number_p_dt_dv">-71579-02-08T15:03:14.552Z name="id">D name="number_p_dt_dv">-117292-05-19T19:23:42.342Z name="id">C name="number_p_dt_dv">-236708-05-08T15:18:34.650Z name="id">Q name="number_p_dt_dv">-279851-11-04T08:31:48.940Z name="id">T name="number_p_dt_dv">-298426-05-18T11:07:08.059Z name="id">L name="number_p_dt_dv">-424243-10-30T19:47:50.864Z name="id">J name="number_p_dt_dv">-500593-12-19T00:44:52.457Z name="id">V name="number_p_dt_dv">-644149-02-10T23:07:16.955Z name="id">W name="number_p_dt_dv">-659321-04-17T04:29:21.261Z name="id">] name="number_p_dt_dv">-771072-10-19T17:00:40.997Z name="id">B name="number_p_dt_dv">-844756-02-08T16:51:18.073Z name="id">^ name="number_p_dt_dv">-844852-04-22T10:39:12.946Z name="id">[ name="number_p_dt_dv">-854949-01-22T23:26:25.473Z name="id">G name="number_p_dt_dv">-867161-10-28T16:28:50.272Z name="id">H name="number_p_dt_dv">-911399-03-28T15:02:37.797Z 

[jira] [Commented] (SOLR-11825) TestPointFields.testDatePointFieldSortAndFunction() failure

[Steve Rowe (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16314078#comment-16314078
 ] 

Steve Rowe commented on SOLR-11825:
---

The issue appears to be that the float sort value isn't precise enough to 
distinguish between two dates that are fairly close.  After adding the sort 
value {{product(-1,ms(field))}} to the {{fl}} list, the two out-of-order docs 
are:

{code:xml}

  F
  +293401-10-30T14:59:03.236Z
  -9.1967007E15


  Y
  +293401-11-02T19:17:28.572Z
  -9.1967007E15

{code}

> TestPointFields.testDatePointFieldSortAndFunction() failure
> ---
>
> Key: SOLR-11825
> URL: https://issues.apache.org/jira/browse/SOLR-11825
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Steve Rowe
>
> Reproducing master seed from 
> [https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/21216/]:
> {noformat}
> Checking out Revision 5a08fa8bbb1cf26b4af5b71549671c31e1427f44 
> (refs/remotes/origin/master)
> [...]
>[junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestPointFields 
> -Dtests.method=testDatePointFieldSortAndFunction 
> -Dtests.seed=A41248828EFF34E3 -Dtests.multiplier=3 -Dtests.slow=true 
> -Dtests.locale=ar-YE -Dtests.timezone=Etc/GMT+10 -Dtests.asserts=true 
> -Dtests.file.encoding=UTF-8
>[junit4] ERROR   0.17s J2 | 
> TestPointFields.testDatePointFieldSortAndFunction <<<
>[junit4]> Throwable #1: java.lang.RuntimeException: Exception during 
> query
>[junit4]>  at 
> __randomizedtesting.SeedInfo.seed([A41248828EFF34E3:73AF779D2D1E9BFD]:0)
>[junit4]>  at 
> org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:902)
>[junit4]>  at 
> org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:862)
>[junit4]>  at 
> org.apache.solr.schema.TestPointFields.doTestDatePointFunctionQuery(TestPointFields.java:3599)
>[junit4]>  at 
> org.apache.solr.schema.TestPointFields.testDatePointFieldSortAndFunction(TestPointFields.java:1664)
>[junit4]>  at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>[junit4]>  at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>[junit4]>  at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>[junit4]>  at 
> java.base/java.lang.reflect.Method.invoke(Method.java:564)
>[junit4]>  at java.base/java.lang.Thread.run(Thread.java:844)
>[junit4]> Caused by: java.lang.RuntimeException: REQUEST FAILED: 
> xpath=//result/doc[9]/date[@name='number_p_dt_dv'][.='+293401-11-02T19:17:28.572Z']
>[junit4]>  xml response was: 
>[junit4]> 
>[junit4]> 0 name="QTime">0 start="0">R name="number_p_dt_dv">+831895-07-23T21:09:09.338Z name="id">Z name="number_p_dt_dv">+725281-03-30T13:09:50.329Z name="id">A name="number_p_dt_dv">+681046-11-04T12:49:38.424Z name="id">S name="number_p_dt_dv">+662906-09-27T18:36:37.903Z name="id">K name="number_p_dt_dv">+477986-01-24T14:48:55.265Z name="id">M name="number_p_dt_dv">+395512-11-01T10:23:52.727Z name="id">I name="number_p_dt_dv">+350980-09-17T07:18:22.252Z name="id">O name="number_p_dt_dv">+295226-09-03T14:25:02.014Z name="id">F name="number_p_dt_dv">+293401-10-30T14:59:03.236Z name="id">Y name="number_p_dt_dv">+293401-11-02T19:17:28.572Z name="id">N name="number_p_dt_dv">+284573-12-19T12:15:27.792Z name="id">X name="number_p_dt_dv">+223248-07-13T00:56:07.425Z name="id">U name="number_p_dt_dv">+13953-10-02T03:24:10.232Z name="id">E name="number_p_dt_dv">-71579-02-08T15:03:14.552Z name="id">D name="number_p_dt_dv">-117292-05-19T19:23:42.342Z name="id">C name="number_p_dt_dv">-236708-05-08T15:18:34.650Z name="id">Q name="number_p_dt_dv">-279851-11-04T08:31:48.940Z name="id">T name="number_p_dt_dv">-298426-05-18T11:07:08.059Z name="id">L name="number_p_dt_dv">-424243-10-30T19:47:50.864Z name="id">J name="number_p_dt_dv">-500593-12-19T00:44:52.457Z name="id">V name="number_p_dt_dv">-644149-02-10T23:07:16.955Z name="id">W name="number_p_dt_dv">-659321-04-17T04:29:21.261Z name="id">] name="number_p_dt_dv">-771072-10-19T17:00:40.997Z name="id">B name="number_p_dt_dv">-844756-02-08T16:51:18.073Z name="id">^ name="number_p_dt_dv">-844852-04-22T10:39:12.946Z name="id">[ name="number_p_dt_dv">-854949-01-22T23:26:25.473Z name="id">G name="number_p_dt_dv">-867161-10-28T16:28:50.272Z name="id">H name="number_p_dt_dv">-911399-03-28T15:02:37.797Z name="id">P name="number_p_dt_dv">-911691-05-13T13:07:48.860Z name="id">\ name="number_p_dt_dv">-921747-04-13T05:12:36.872Z
>[junit4]> 
>[junit4]>  request 
> 

[jira] [Updated] (SOLR-10716) Add termVectors Stream Evaluator

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-10716:
--
Description: 
The termVectors Stream Evaluator returns tf-idf word vectors for a text field 
in a list of tuples. 

Syntax:
{code}
let(a=select(search(...), analyze(a, body) as terms),
 b=termVectors(a, minDocFreq=".00", maxDocFreq="1.0")) 
{code}

The code above performs a search then uses the *select* stream and *analyze* 
evaluator to attach a list of terms to each document.

 

  was:
The termVectors Stream Evaluator returns tf-idf word vectors for a text field 
in a list of tuples. A Lucene analyzer can be specified to support flexible 
word analysis.

The word vectors can then be used for various machine learning operations.

Syntax:
{code}
r = search()
v = termVectors(r, fieldA, analyzerFied)
{code}

 


> Add termVectors Stream Evaluator
> 
>
> Key: SOLR-10716
> URL: https://issues.apache.org/jira/browse/SOLR-10716
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: streaming expressions
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
> Fix For: 7.0
>
> Attachments: SOLR-10716.patch, SOLR-10716.patch
>
>
> The termVectors Stream Evaluator returns tf-idf word vectors for a text field 
> in a list of tuples. 
> Syntax:
> {code}
> let(a=select(search(...), analyze(a, body) as terms),
>  b=termVectors(a, minDocFreq=".00", maxDocFreq="1.0")) 
> {code}
> The code above performs a search then uses the *select* stream and *analyze* 
> evaluator to attach a list of terms to each document.
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10716) Add termVectors Stream Evaluator

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-10716:
--
Attachment: SOLR-10716.patch

New patch

> Add termVectors Stream Evaluator
> 
>
> Key: SOLR-10716
> URL: https://issues.apache.org/jira/browse/SOLR-10716
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: streaming expressions
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
> Fix For: 7.0
>
> Attachments: SOLR-10716.patch, SOLR-10716.patch
>
>
> The termVectors Stream Evaluator returns tf-idf word vectors for a text field 
> in a list of tuples. A Lucene analyzer can be specified to support flexible 
> word analysis.
> The word vectors can then be used for various machine learning operations.
> Syntax:
> {code}
> r = search()
> v = termVectors(r, fieldA, analyzerFied)
> {code}
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk-9) - Build # 4368 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/4368/
Java: 64bit/jdk-9 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

3 tests failed.
FAILED:  
org.apache.solr.cloud.autoscaling.sim.TestExecutePlanAction.testIntegration

Error Message:


Stack Trace:
java.util.ConcurrentModificationException
at 
__randomizedtesting.SeedInfo.seed([CB0D016BA32E97B5:7B6C0F4786113690]:0)
at 
java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:937)
at java.base/java.util.ArrayList$Itr.next(ArrayList.java:891)
at 
org.apache.solr.cloud.autoscaling.sim.SimSolrCloudTestCase.tearDown(SimSolrCloudTestCase.java:141)
at jdk.internal.reflect.GeneratedMethodAccessor29.invoke(Unknown Source)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  
org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeAddedTriggerRestoreState

Error Message:


Stack Trace:
java.util.ConcurrentModificationException
at 
__randomizedtesting.SeedInfo.seed([CB0D016BA32E97B5:4330881499EE7618]:0)
at 
java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:937)
at java.base/java.util.ArrayList$Itr.next(ArrayList.java:891)
at 
org.apache.solr.cloud.autoscaling.sim.SimSolrCloudTestCase.tearDown(SimSolrCloudTestCase.java:141)
at 

[jira] [Updated] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log

[Steve Rowe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Rowe updated LUCENE-8106:
---
Attachment: LUCENE-8106.patch

Updated patch, adding a report at the end, e.g.:

{noformat}
[repro] Failures:
[repro]   0/5 failed: 
org.apache.solr.client.solrj.io.stream.SelectWithEvaluatorsTest
[repro]   0/5 failed: 
org.apache.solr.cloud.autoscaling.sim.TestComputePlanAction
[repro]   5/5 failed: org.apache.solr.schema.TestPointFields
{noformat}

I think it's ready.  I'll commit in a couple days if there are no objections.

> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Steve Rowe
> Attachments: LUCENE-8106.patch, LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream 
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
>  python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-11825) TestPointFields.testDatePointFieldSortAndFunction() failure

[Steve Rowe (JIRA)]

Steve Rowe created SOLR-11825:
-

 Summary: TestPointFields.testDatePointFieldSortAndFunction() 
failure
 Key: SOLR-11825
 URL: https://issues.apache.org/jira/browse/SOLR-11825
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Steve Rowe


Reproducing master seed from 
[https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/21216/]:

{noformat}
Checking out Revision 5a08fa8bbb1cf26b4af5b71549671c31e1427f44 
(refs/remotes/origin/master)
[...]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestPointFields 
-Dtests.method=testDatePointFieldSortAndFunction -Dtests.seed=A41248828EFF34E3 
-Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ar-YE 
-Dtests.timezone=Etc/GMT+10 -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.17s J2 | 
TestPointFields.testDatePointFieldSortAndFunction <<<
   [junit4]> Throwable #1: java.lang.RuntimeException: Exception during 
query
   [junit4]>at 
__randomizedtesting.SeedInfo.seed([A41248828EFF34E3:73AF779D2D1E9BFD]:0)
   [junit4]>at 
org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:902)
   [junit4]>at 
org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:862)
   [junit4]>at 
org.apache.solr.schema.TestPointFields.doTestDatePointFunctionQuery(TestPointFields.java:3599)
   [junit4]>at 
org.apache.solr.schema.TestPointFields.testDatePointFieldSortAndFunction(TestPointFields.java:1664)
   [junit4]>at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]>at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]>at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]>at 
java.base/java.lang.reflect.Method.invoke(Method.java:564)
   [junit4]>at java.base/java.lang.Thread.run(Thread.java:844)
   [junit4]> Caused by: java.lang.RuntimeException: REQUEST FAILED: 
xpath=//result/doc[9]/date[@name='number_p_dt_dv'][.='+293401-11-02T19:17:28.572Z']
   [junit4]>xml response was: 
   [junit4]> 
   [junit4]> 00R+831895-07-23T21:09:09.338ZZ+725281-03-30T13:09:50.329ZA+681046-11-04T12:49:38.424ZS+662906-09-27T18:36:37.903ZK+477986-01-24T14:48:55.265ZM+395512-11-01T10:23:52.727ZI+350980-09-17T07:18:22.252ZO+295226-09-03T14:25:02.014ZF+293401-10-30T14:59:03.236ZY+293401-11-02T19:17:28.572ZN+284573-12-19T12:15:27.792ZX+223248-07-13T00:56:07.425ZU+13953-10-02T03:24:10.232ZE-71579-02-08T15:03:14.552ZD-117292-05-19T19:23:42.342ZC-236708-05-08T15:18:34.650ZQ-279851-11-04T08:31:48.940ZT-298426-05-18T11:07:08.059ZL-424243-10-30T19:47:50.864ZJ-500593-12-19T00:44:52.457ZV-644149-02-10T23:07:16.955ZW-659321-04-17T04:29:21.261Z]-771072-10-19T17:00:40.997ZB-844756-02-08T16:51:18.073Z^-844852-04-22T10:39:12.946Z[-854949-01-22T23:26:25.473ZG-867161-10-28T16:28:50.272ZH-911399-03-28T15:02:37.797ZP-911691-05-13T13:07:48.860Z\-921747-04-13T05:12:36.872Z
   [junit4]> 
   [junit4]>request 
was:q=*:*=id,+number_p_dt_dv=product(-1,ms(number_p_dt_dv))+asc=30=xml
   [junit4]>at 
org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:895)
[...]
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): 
{id=PostingsFormat(name=Memory)}, 
docValues:{foo_p_f_ni_dv_ns=DocValuesFormat(name=Lucene70), 
foo_p_f_ni_dv_ns_mv=DocValuesFormat(name=Lucene70), 
number_p_dt_dv_ns=DocValuesFormat(name=Asserting), 
foo_p_d_ni_dv_ns_mv=DocValuesFormat(name=Lucene70), 
foo_p_i_ni_dv_ns=DocValuesFormat(name=Asserting), 
number_p_f_ni_mv_dv_smf=DocValuesFormat(name=Lucene70), 
number_p_dt_ni_mv_dv_sml=DocValuesFormat(name=Lucene70), 
number_p_f_dv=DocValuesFormat(name=Asserting), 
number_p_i_dv_ns=DocValuesFormat(name=Lucene70), 
number_p_l_ni_dv=DocValuesFormat(name=Asserting), 
number_p_l_dv_ns=DocValuesFormat(name=Asserting), 
foo_p_l_ni_dv_ns=DocValuesFormat(name=Lucene70), 
number_p_dt_ni_mv_dv=DocValuesFormat(name=Asserting), 
number_p_l_dv_sml=DocValuesFormat(name=Lucene70), 
number_p_i_dv_smf=DocValuesFormat(name=Direct), 
number_p_d_ni_ns_dv=DocValuesFormat(name=Direct), 
number_p_dt_ni_dv_ns_mv=DocValuesFormat(name=Asserting), 
number_p_i_ni_dv_ns_mv=DocValuesFormat(name=Lucene70), 
number_p_f_dv_smf=DocValuesFormat(name=Lucene70), 
number_p_l_ni_dv_ns=DocValuesFormat(name=Direct), 
number_p_l_dv_smf=DocValuesFormat(name=Lucene70), 
number_p_d_ni_mv_dv_sml=DocValuesFormat(name=Lucene70), 
number_p_d_dv_smf=DocValuesFormat(name=Lucene70), 
number_p_f_ni_dv_ns_mv=DocValuesFormat(name=Direct), 
number_p_dt_ni_dv=DocValuesFormat(name=Asserting), 
number_p_f_ni_mv_dv_sml=DocValuesFormat(name=Lucene70), 
number_p_d_ni_mv_dv_smf=DocValuesFormat(name=Lucene70), 

[JENKINS-EA] Lucene-Solr-7.x-Linux (64bit/jdk-10-ea+37) - Build # 1126 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/1126/
Java: 64bit/jdk-10-ea+37 -XX:+UseCompressedOops -XX:+UseG1GC

4 tests failed.
FAILED:  org.apache.solr.cloud.ShardSplitTest.testSplitShardWithRule

Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:40005

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting 
response from server at: http://127.0.0.1:40005
at 
__randomizedtesting.SeedInfo.seed([6FF05F542DB4490A:EEFE8EE6B0461221]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:654)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:320)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:991)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  

Indexing fingerprinting and PeerSync

[Varun Thacker]

Hi Everyone,

I was looking into a scenario where PeerSync failed even when we had a high
number maxNumLogsToKeep ( 200 ) and numRecordsToKeep ( 20 )

The log excerpt is at
https://gist.github.com/vthacker/fb536c6f1146dd0d7513afb9960a10e3 and I am
still trying to pinpoint the actual cause . It looks to me that the replica
has more number of documents till that version ( numVersions ) than the
leader and I can't tell why. Does this look like a bug?

While trying to reproduce it locally here is one scenario that I ran into :

   1. I kept a very low numRecordsToKeep ( 5 ) . Indexed like 3 or 4 docs
   while the replica was down and then started it up. PeerSync failed because
   of
   
https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/update/PeerSync.java#L655
   . Do we need to do a threshold check when we are verifying via
   fingerprinting if the indexes are the same or not? From my understanding we
   can avoid this check when fingerprinting is enabled but wanted to check
   before filing a Jira

[JENKINS] Lucene-Solr-7.x-Windows (64bit/jdk1.8.0_144) - Build # 383 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Windows/383/
Java: 64bit/jdk1.8.0_144 -XX:+UseCompressedOops -XX:+UseParallelGC

8 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.lucene.analysis.miscellaneous.TestWordDelimiterFilter

Error Message:
Could not remove the following files (in the order of attempts):
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\analysis\common\test\J1\temp\lucene.analysis.miscellaneous.TestWordDelimiterFilter_D420BB736B74E8AF-001\bttc-001:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\analysis\common\test\J1\temp\lucene.analysis.miscellaneous.TestWordDelimiterFilter_D420BB736B74E8AF-001\bttc-001

C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\analysis\common\test\J1\temp\lucene.analysis.miscellaneous.TestWordDelimiterFilter_D420BB736B74E8AF-001\bttc-001\_3.si:
 java.nio.file.AccessDeniedException: 
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\analysis\common\test\J1\temp\lucene.analysis.miscellaneous.TestWordDelimiterFilter_D420BB736B74E8AF-001\bttc-001\_3.si

C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\analysis\common\test\J1\temp\lucene.analysis.miscellaneous.TestWordDelimiterFilter_D420BB736B74E8AF-001:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\analysis\common\test\J1\temp\lucene.analysis.miscellaneous.TestWordDelimiterFilter_D420BB736B74E8AF-001
 

Stack Trace:
java.io.IOException: Could not remove the following files (in the order of 
attempts):
   
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\analysis\common\test\J1\temp\lucene.analysis.miscellaneous.TestWordDelimiterFilter_D420BB736B74E8AF-001\bttc-001:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\analysis\common\test\J1\temp\lucene.analysis.miscellaneous.TestWordDelimiterFilter_D420BB736B74E8AF-001\bttc-001
   
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\analysis\common\test\J1\temp\lucene.analysis.miscellaneous.TestWordDelimiterFilter_D420BB736B74E8AF-001\bttc-001\_3.si:
 java.nio.file.AccessDeniedException: 
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\analysis\common\test\J1\temp\lucene.analysis.miscellaneous.TestWordDelimiterFilter_D420BB736B74E8AF-001\bttc-001\_3.si
   
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\analysis\common\test\J1\temp\lucene.analysis.miscellaneous.TestWordDelimiterFilter_D420BB736B74E8AF-001:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\analysis\common\test\J1\temp\lucene.analysis.miscellaneous.TestWordDelimiterFilter_D420BB736B74E8AF-001

at __randomizedtesting.SeedInfo.seed([D420BB736B74E8AF]:0)
at org.apache.lucene.util.IOUtils.rm(IOUtils.java:329)
at 
org.apache.lucene.util.TestRuleTemporaryFilesCleanup.afterAlways(TestRuleTemporaryFilesCleanup.java:216)
at 
com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.afterAlways(TestRuleAdapter.java:31)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)


FAILED:  
junit.framework.TestSuite.org.apache.lucene.store.TestSingleInstanceLockFactory

Error Message:
Could not remove the following files (in the order of attempts):
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\core\test\J0\temp\lucene.store.TestSingleInstanceLockFactory_95790CC048D43ECB-001\tempDir-006:
 java.nio.file.AccessDeniedException: 
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\core\test\J0\temp\lucene.store.TestSingleInstanceLockFactory_95790CC048D43ECB-001\tempDir-006

C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\core\test\J0\temp\lucene.store.TestSingleInstanceLockFactory_95790CC048D43ECB-001:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\core\test\J0\temp\lucene.store.TestSingleInstanceLockFactory_95790CC048D43ECB-001
 

Stack Trace:

[jira] [Resolved] (SOLR-4025) queryAnalyzerFieldType for spellcheck is never validated, makes example configs broken

[Cassandra Targett (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-4025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Cassandra Targett resolved SOLR-4025.
-
Resolution: Cannot Reproduce

This isn't a problem anymore (using 7.2) since the default spell check 
component config (for both sample configsets) has {{text_general}}, and that field type has the 
LowerCaseFilterFactory configured for both index and query analysis.

> queryAnalyzerFieldType for spellcheck is never validated, makes example 
> configs broken
> --
>
> Key: SOLR-4025
> URL: https://issues.apache.org/jira/browse/SOLR-4025
> Project: Solr
>  Issue Type: Bug
>Reporter: Hoss Man
>
> Discovered this while trying to answer a quick question for "nepenthe" on the 
> IRC channel...
> * spellcheck component looks for an init param named "queryAnalyzerFieldType" 
> butsilently ignores it and uses WhitespaceAnalyzer if it refers to a 
> fieldtype that doesn't exist.
> * as a result of silently ignoring this w/o failure, no one seems to have 
> ever noticed that the example configs refer to a field type that was 
> removed...
> {noformat}
> textSpell
> {noformat}
> * as a result, the "default" example spellchecker dictionary (using 
> DirectSolrSpellChecker) doesn't give any recomendations for words with 
> upercase letters (because it defaults to WhitespaceAnalyzer but the "name" 
> field used for hte dictionary uses LowercaseFilter)
> For example, this gives no suggestions...
> http://localhost:8983/solr/spell?q=Delll+Ultrashar=true=true=default=true
> But this does...
> http://localhost:8983/solr/spell?q=delll+ultrashar=true=true=default=true



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Closed] (SOLR-4025) queryAnalyzerFieldType for spellcheck is never validated, makes example configs broken

[Cassandra Targett (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-4025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Cassandra Targett closed SOLR-4025.
---

> queryAnalyzerFieldType for spellcheck is never validated, makes example 
> configs broken
> --
>
> Key: SOLR-4025
> URL: https://issues.apache.org/jira/browse/SOLR-4025
> Project: Solr
>  Issue Type: Bug
>Reporter: Hoss Man
>
> Discovered this while trying to answer a quick question for "nepenthe" on the 
> IRC channel...
> * spellcheck component looks for an init param named "queryAnalyzerFieldType" 
> butsilently ignores it and uses WhitespaceAnalyzer if it refers to a 
> fieldtype that doesn't exist.
> * as a result of silently ignoring this w/o failure, no one seems to have 
> ever noticed that the example configs refer to a field type that was 
> removed...
> {noformat}
> textSpell
> {noformat}
> * as a result, the "default" example spellchecker dictionary (using 
> DirectSolrSpellChecker) doesn't give any recomendations for words with 
> upercase letters (because it defaults to WhitespaceAnalyzer but the "name" 
> field used for hte dictionary uses LowercaseFilter)
> For example, this gives no suggestions...
> http://localhost:8983/solr/spell?q=Delll+Ultrashar=true=true=default=true
> But this does...
> http://localhost:8983/solr/spell?q=delll+ultrashar=true=true=default=true



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-4039) MergeIndex on multiple cores impossible with SolrJ

[Cassandra Targett (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-4039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Cassandra Targett updated SOLR-4039:

Component/s: clients - java

> MergeIndex on multiple cores impossible with SolrJ
> --
>
> Key: SOLR-4039
> URL: https://issues.apache.org/jira/browse/SOLR-4039
> Project: Solr
>  Issue Type: Bug
>  Components: clients - java
>Affects Versions: 3.6.1
> Environment: Windows
>Reporter: Mathieu Gond
>Assignee: Mark Miller
> Fix For: 4.9, 6.0
>
>
> It is not possible to do a mergeIndexes action on multiple cores at the same 
> time with SolrJ.
> Only the last core set in the srcCores parameter is used.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-4006) Many tests on Apache Jenkins are failing with lingering threads.

[Cassandra Targett (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-4006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Cassandra Targett updated SOLR-4006:

Component/s: Tests

> Many tests on Apache Jenkins are failing with lingering threads.
> 
>
> Key: SOLR-4006
> URL: https://issues.apache.org/jira/browse/SOLR-4006
> Project: Solr
>  Issue Type: Bug
>  Components: Tests
>Reporter: Mark Miller
>
> I think I've tracked this down to being related to the black hole.
> It seems to be a recovery call to a server that is down or something - it's 
> hanging in the connect method even though we are using a connect timeout.
> {noformat}
> Thread[RecoveryThread,5,TGRP-SyncSliceTest]
> java.net.PlainSocketImpl.socketConnect(Native Method)
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:327)
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:193)
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:180)
> java.net.SocksSocketImpl.connect(SocksSocketImpl.java:384)
> java.net.Socket.connect(Socket.java:546)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Linux (64bit/jdk1.8.0_144) - Build # 21220 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/21220/
Java: 64bit/jdk1.8.0_144 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

4 tests failed.
FAILED:  org.apache.solr.cloud.TestPullReplica.testKillLeader

Error Message:
Error from server at https://127.0.0.1:33041/solr: create the collection time 
out:180s

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at https://127.0.0.1:33041/solr: create the collection time out:180s
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at 
org.apache.solr.cloud.TestPullReplica.doTestNoLeader(TestPullReplica.java:372)
at 
org.apache.solr.cloud.TestPullReplica.testKillLeader(TestPullReplica.java:290)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[jira] [Updated] (SOLR-3218) Range faceting support for CurrencyField

[Hoss Man (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-3218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hoss Man updated SOLR-3218:
---
Attachment: SOLR-3218.patch

Updated patch with a lot more {{json.facet}} tests.

While working on this i discovered SOLR-11824 -- these tests will currently 
fail very badly w/o the patch from SOLR-11824 also applied.

There is also still one test logic flaw in {{testJsonRangeFacetAsSubFacet}} 
because of SOLR-11733 ... i thought i had been creative enough in the 
documents/queries used to ensure that we'd get a consistent "top term" for 
these facets -- but I released later that with random sharding it's still 
possible to "miss" the term the test expects due to it being in the "long tail" 
... need to fix that.

bq. ...more notably because of the following points of confusion I have about 
the code in the existing patch that I'm hoping Yonik can provide some guidance 
on...

FWIW: I still haven't wrapped my head around these 2 questions.  Hoping i 
trigger some of these {{throw new RuntimeException("nocommit:...}} as i keep 
writing tests

> Range faceting support for CurrencyField
> 
>
> Key: SOLR-3218
> URL: https://issues.apache.org/jira/browse/SOLR-3218
> Project: Solr
>  Issue Type: Improvement
>  Components: Schema and Analysis
>Reporter: Jan Høydahl
>Assignee: Hoss Man
> Attachments: SOLR-3218-1.patch, SOLR-3218-2.patch, SOLR-3218.patch, 
> SOLR-3218.patch, SOLR-3218.patch, SOLR-3218.patch, SOLR-3218.patch, 
> SOLR-3218.patch, SOLR-3218.patch, SOLR-3218.patch
>
>
> Spinoff from SOLR-2202. Need to add range faceting capabilities for 
> CurrencyField



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11824) distributed json.facet "type:range" can return buckets out of order when mincount>0

[Hoss Man (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hoss Man updated SOLR-11824:

Attachment: SOLR-11824.patch


Here's a patch with beefed up testing to trigger this, and the simplest 
possible fix I know of (borrowing from SOLR-6154): the shards should ignore the 
mincount and allways return every bucket.  This way, regardless of which shard 
replies first, we always know all of the potential buckets (in the correct 
order) and the other shard responses can be merged in trivially.

The other potential approach is to let the shards force a {{mincount=1}} (like 
the existing broken code) and then re-sort the (out of order) buckets -- either 
as they arrive or after all shards have reponded -- but for things like 
CurrencyField SOLR-3218 that's non trivial because the {{"val"}} keys on the 
buckets aren't trivially sortable.  We'd have to complicate the 
{{FacetRangeMerger}} with knowledge of the {{FacetRange.Calc}} ... this seems 
much simpler.

---

Any concerns?

> distributed json.facet "type:range" can return buckets out of order when 
> mincount>0
> ---
>
> Key: SOLR-11824
> URL: https://issues.apache.org/jira/browse/SOLR-11824
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Facet Module
>Reporter: Hoss Man
> Attachments: SOLR-11824.patch
>
>
> Discovered this while working on SOLR-3218, but it's a general problem with 
> how the code is designed (and the existing code even has a TODO in 
> FacetRangeMerger refering to the problem) ... this is essentially the same 
> bug FacetComponent's {{facet.range}} had that was fixed in SOLR-6154: When he 
> coordinator "combines" the response from each shard, the buckets can get out 
> of order depending on which buckets were excluded by the first shard to 
> respond.
> Depending on how many shards you have, it's fairly trivial to reproduce using 
> {{bin/solr -e cloud}} and the {{exampledocs/*.xml}} files...
> {noformat}
> $ curl http://localhost:8983/solr/techproducts/select -d 
> 'q=*:*=0=xml={
> foo:{ type:range, field:price, start:0, end:2000, gap:100, other:all, 
> mincount:1} }'
> 
> 
> 
>   true
>   0
>   13
>   
> *:*
> {
> foo:{ type:range, field:price, start:0, end:2000, gap:100, other:all, 
> mincount:1} }
> 0
> xml
>   
> 
> 
> 
> 
>   32
>   
> 
>   
> 0.0
> 7
>   
>   
> 100.0
> 2
>   
>   
> 300.0
> 3
>   
>   
> 400.0
> 1
>   
>   
> 200.0
> 1
>   
>   
> 600.0
> 1
>   
> 
> 
>   0
> 
> 
>   1
> 
> 
>   15
> 
>   
> 
> 
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8118) ArrayIndexOutOfBoundsException in TermsHashPerField.writeByte during indexing

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16313744#comment-16313744
 ] 

Robert Muir commented on LUCENE-8118:
-

the test had to work hard to hit AIOOBE instead of OOM. 

I think most users that do something like this will hit OOM which is just as 
confusing and bad. it may technically be a different problem but due to the 
names of the methods and the apis, i think its easy someone will hit it too. 
Seems like add/updateDocuments need some sanity checks...

> ArrayIndexOutOfBoundsException in TermsHashPerField.writeByte during indexing
> -
>
> Key: LUCENE-8118
> URL: https://issues.apache.org/jira/browse/LUCENE-8118
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index
>Affects Versions: 7.2
> Environment: Debian/Stretch
> java version "1.8.0_144"  
>   
>Java(TM) SE Runtime 
> Environment (build 1.8.0_144-b01) 
>   
>Java HotSpot(TM) 64-Bit Server VM (build 
> 25.144-b01, mixed mode)
>Reporter: Laura Dietz
> Attachments: LUCENE-8118_test.patch
>
>
> Indexing a large collection of about 20 million paragraph-sized documents 
> results in an ArrayIndexOutOfBoundsException in 
> org.apache.lucene.index.TermsHashPerField.writeByte  (full stack trace 
> below). 
> The bug is possibly related to issues described in 
> [here|http://lucene.472066.n3.nabble.com/ArrayIndexOutOfBoundsException-65536-td3661945.html]
>   and [SOLR-10936|https://issues.apache.org/jira/browse/SOLR-10936] -- but I 
> am not using SOLR, I am directly using Lucene Core.
> The issue can be reproduced using code from  [GitHub 
> trec-car-tools-example|https://github.com/TREMA-UNH/trec-car-tools/tree/lucene-bug/trec-car-tools-example]
>  
> - compile with `mvn compile assembly:single`
> - run with `java -cp 
> ./target/treccar-tools-example-0.1-jar-with-dependencies.jar 
> edu.unh.cs.TrecCarBuildLuceneIndex paragraphs paragraphCorpus.cbor indexDir`
> Where paragraphCorpus.cbor is contained in this 
> [archive|http://trec-car.cs.unh.edu/datareleases/v2.0-snapshot/archive-paragraphCorpus.tar.xz]
> Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -65536   
> at 
> org.apache.lucene.index.TermsHashPerField.writeByte(TermsHashPerField.java:198)
>   
>at 
> org.apache.lucene.index.TermsHashPerField.writeVInt(TermsHashPerField.java:224)
>   
>at 
> org.apache.lucene.index.FreqProxTermsWriterPerField.addTerm(FreqProxTermsWriterPerField.java:159)
>   
>  at 
> org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:185) 
>   
> at 
> org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:786)
>   
>at 
> org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:430)
>   
>   at 
> org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:392)
>   
>at 
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:281)
>   
>at 
> org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:451)
>   
>  at 
> org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1532)
>   
> at 
> 

[jira] [Assigned] (SOLR-11824) distributed json.facet "type:range" can return buckets out of order when mincount>0

[Hoss Man (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hoss Man reassigned SOLR-11824:
---

Assignee: Hoss Man

> distributed json.facet "type:range" can return buckets out of order when 
> mincount>0
> ---
>
> Key: SOLR-11824
> URL: https://issues.apache.org/jira/browse/SOLR-11824
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Facet Module
>Reporter: Hoss Man
>Assignee: Hoss Man
> Attachments: SOLR-11824.patch
>
>
> Discovered this while working on SOLR-3218, but it's a general problem with 
> how the code is designed (and the existing code even has a TODO in 
> FacetRangeMerger refering to the problem) ... this is essentially the same 
> bug FacetComponent's {{facet.range}} had that was fixed in SOLR-6154: When he 
> coordinator "combines" the response from each shard, the buckets can get out 
> of order depending on which buckets were excluded by the first shard to 
> respond.
> Depending on how many shards you have, it's fairly trivial to reproduce using 
> {{bin/solr -e cloud}} and the {{exampledocs/*.xml}} files...
> {noformat}
> $ curl http://localhost:8983/solr/techproducts/select -d 
> 'q=*:*=0=xml={
> foo:{ type:range, field:price, start:0, end:2000, gap:100, other:all, 
> mincount:1} }'
> 
> 
> 
>   true
>   0
>   13
>   
> *:*
> {
> foo:{ type:range, field:price, start:0, end:2000, gap:100, other:all, 
> mincount:1} }
> 0
> xml
>   
> 
> 
> 
> 
>   32
>   
> 
>   
> 0.0
> 7
>   
>   
> 100.0
> 2
>   
>   
> 300.0
> 3
>   
>   
> 400.0
> 1
>   
>   
> 200.0
> 1
>   
>   
> 600.0
> 1
>   
> 
> 
>   0
> 
> 
>   1
> 
> 
>   15
> 
>   
> 
> 
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-11824) distributed json.facet "type:range" can return buckets out of order when mincount>0

[Hoss Man (JIRA)]

Hoss Man created SOLR-11824:
---

 Summary: distributed json.facet "type:range" can return buckets 
out of order when mincount>0
 Key: SOLR-11824
 URL: https://issues.apache.org/jira/browse/SOLR-11824
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
  Components: Facet Module
Reporter: Hoss Man


Discovered this while working on SOLR-3218, but it's a general problem with how 
the code is designed (and the existing code even has a TODO in FacetRangeMerger 
refering to the problem) ... this is essentially the same bug FacetComponent's 
{{facet.range}} had that was fixed in SOLR-6154: When he coordinator "combines" 
the response from each shard, the buckets can get out of order depending on 
which buckets were excluded by the first shard to respond.

Depending on how many shards you have, it's fairly trivial to reproduce using 
{{bin/solr -e cloud}} and the {{exampledocs/*.xml}} files...

{noformat}
$ curl http://localhost:8983/solr/techproducts/select -d 
'q=*:*=0=xml={
foo:{ type:range, field:price, start:0, end:2000, gap:100, other:all, 
mincount:1} }'




  true
  0
  13
  
*:*
{
foo:{ type:range, field:price, start:0, end:2000, gap:100, other:all, 
mincount:1} }
0
xml
  




  32
  

  
0.0
7
  
  
100.0
2
  
  
300.0
3
  
  
400.0
1
  
  
200.0
1
  
  
600.0
1
  


  0


  1


  15

  


{noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-8120) Fix LatLonBoundingBox's toString() method

[Martijn van Groningen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-8120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Martijn van Groningen updated LUCENE-8120:
--
Attachment: LUCENE-8120.patch

> Fix LatLonBoundingBox's toString() method
> -
>
> Key: LUCENE-8120
> URL: https://issues.apache.org/jira/browse/LUCENE-8120
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Martijn van Groningen
>Priority: Trivial
> Attachments: LUCENE-8120.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-8120) Fix LatLonBoundingBox's toString() method

[Martijn van Groningen (JIRA)]

Martijn van Groningen created LUCENE-8120:
-

 Summary: Fix LatLonBoundingBox's toString() method
 Key: LUCENE-8120
 URL: https://issues.apache.org/jira/browse/LUCENE-8120
 Project: Lucene - Core
  Issue Type: Bug
Reporter: Martijn van Groningen
Priority: Trivial






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-11653) create next time collection based on a fixed time gap

[David Smiley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Smiley resolved SOLR-11653.
-
   Resolution: Fixed
Fix Version/s: 7.3

Thanks for the review Gus.  With this committed you can now reference some of 
the utility methods in SOLR-11722.

> create next time collection based on a fixed time gap
> -
>
> Key: SOLR-11653
> URL: https://issues.apache.org/jira/browse/SOLR-11653
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
>Assignee: David Smiley
> Fix For: 7.3
>
> Attachments: SOLR-11653.patch, SOLR-11653.patch, SOLR-11653.patch
>
>
> For time series collections (as part of a collection Alias with certain 
> metadata), we want to automatically add new collections. In this issue, this 
> is about creating the next collection based on a configurable fixed time gap. 
>  And we will also add this collection synchronously once a document flowing 
> through the URP chain exceeds the gap, as opposed to asynchronously in 
> advance.  There will be some Alias metadata to define in this issue.  The 
> preponderance of the implementation will be in TimePartitionedUpdateProcessor 
> or perhaps a helper to this URP.
> note: other issues will implement pre-emptive creation and capping 
> collections by size.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11653) create next time collection based on a fixed time gap

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16313669#comment-16313669
 ] 

ASF subversion and git services commented on SOLR-11653:


Commit c59db0c33778bac7430aa4c2dfd0eb39ef60e205 in lucene-solr's branch 
refs/heads/branch_7x from [~dsmiley]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=c59db0c ]

SOLR-11653: TimeRoutedAlias URP now auto-creates collections using new 
RoutedAliasCreateCollectionCmd

(cherry picked from commit 925733d)


> create next time collection based on a fixed time gap
> -
>
> Key: SOLR-11653
> URL: https://issues.apache.org/jira/browse/SOLR-11653
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
>Assignee: David Smiley
> Attachments: SOLR-11653.patch, SOLR-11653.patch, SOLR-11653.patch
>
>
> For time series collections (as part of a collection Alias with certain 
> metadata), we want to automatically add new collections. In this issue, this 
> is about creating the next collection based on a configurable fixed time gap. 
>  And we will also add this collection synchronously once a document flowing 
> through the URP chain exceeds the gap, as opposed to asynchronously in 
> advance.  There will be some Alias metadata to define in this issue.  The 
> preponderance of the implementation will be in TimePartitionedUpdateProcessor 
> or perhaps a helper to this URP.
> note: other issues will implement pre-emptive creation and capping 
> collections by size.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11653) create next time collection based on a fixed time gap

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16313666#comment-16313666
 ] 

ASF subversion and git services commented on SOLR-11653:


Commit 925733d1ef3ac6fbabc450804511c65a4c6424ac in lucene-solr's branch 
refs/heads/master from [~dsmiley]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=925733d ]

SOLR-11653: TimeRoutedAlias URP now auto-creates collections using new 
RoutedAliasCreateCollectionCmd


> create next time collection based on a fixed time gap
> -
>
> Key: SOLR-11653
> URL: https://issues.apache.org/jira/browse/SOLR-11653
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
>Assignee: David Smiley
> Attachments: SOLR-11653.patch, SOLR-11653.patch, SOLR-11653.patch
>
>
> For time series collections (as part of a collection Alias with certain 
> metadata), we want to automatically add new collections. In this issue, this 
> is about creating the next collection based on a configurable fixed time gap. 
>  And we will also add this collection synchronously once a document flowing 
> through the URP chain exceeds the gap, as opposed to asynchronously in 
> advance.  There will be some Alias metadata to define in this issue.  The 
> preponderance of the implementation will be in TimePartitionedUpdateProcessor 
> or perhaps a helper to this URP.
> note: other issues will implement pre-emptive creation and capping 
> collections by size.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-7.x-Linux (32bit/jdk1.8.0_144) - Build # 1125 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/1125/
Java: 32bit/jdk1.8.0_144 -client -XX:+UseConcMarkSweepGC

6 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestRebalanceLeaders

Error Message:
44 threads leaked from SUITE scope at 
org.apache.solr.cloud.TestRebalanceLeaders: 1) Thread[id=13271, 
name=org.eclipse.jetty.server.session.HashSessionManager@ce7d8fTimer, 
state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at 
sun.misc.Unsafe.park(Native Method) at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) 
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
 at 
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
 at 
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
 at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)   
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) 
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
at java.lang.Thread.run(Thread.java:748)2) Thread[id=13264, 
name=qtp23947876-13264, state=RUNNABLE, group=TGRP-TestRebalanceLeaders]
 at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at 
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at 
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at 
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at 
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at 
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at 
org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
 at 
org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
 at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
 at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
 at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
 at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
 at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) 
at java.lang.Thread.run(Thread.java:748)3) Thread[id=13300, 
name=qtp3197479-13300, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders]
 at java.lang.Thread.sleep(Native Method) at 
org.apache.solr.cloud.ZkController.waitForShardId(ZkController.java:1562)   
  at 
org.apache.solr.cloud.ZkController.doGetShardIdAndNodeNameProcess(ZkController.java:1513)
 at 
org.apache.solr.cloud.ZkController.preRegister(ZkController.java:1622) 
at 
org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1032)
 at org.apache.solr.core.CoreContainer.create(CoreContainer.java:950)   
  at 
org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91)
 at 
org.apache.solr.handler.admin.CoreAdminOperation$$Lambda$85/23120768.execute(Unknown
 Source) at 
org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384)
 at 
org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389)
 at 
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174)
 at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
 at 
org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:736) 
at 
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:717)  
   at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:498)  
   at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
 at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
 at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
 at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
 at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
 at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)  
   at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
 at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
 at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)   
  at 

[jira] [Commented] (SOLR-11822) Unable to create core in 7.2

[Erick Erickson (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16313583#comment-16313583
 ] 

Erick Erickson commented on SOLR-11822:
---

Please raise questions like this on the user's list first to be sure it's 
really a bug and not a user error. 

And when you post over there, give us the exact command you used. Are you 
trying to use the core admin API to create a core in a SolrCloud setup? Or did 
you try some collections API action that failed? If the former that's totally 
not recommended, use the Collections API ADDREPLICA command instead.

If you're really trying this with stand-alone, then you need to pre-populate 
the core's directory with the configuration or use a configset, but if the 
Overseer is involved, it must be SolrCloud.

> Unable to create core in 7.2 
> -
>
> Key: SOLR-11822
> URL: https://issues.apache.org/jira/browse/SOLR-11822
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 7.2
> Environment: Ubuntu 16 & Centos 7
>Reporter: Mohandoss
>
> 2018-01-05 08:34:37.496 INFO  (qtp2009221452-22) [   ] o.a.s.s.HttpSolrCall 
> [admin] webapp=null path=/admin/cores 
> params={schema=schema.xml=data=profidx=CREATE=solrconfig.xml=profidx=json&_=1515141248780}
>  status=400 QTime=68
> 2018-01-05 08:44:30.323 INFO  (qtp2009221452-13) [c:profidx   x:profidx] 
> o.a.s.c.ZkController The core 'profidx' had failed to initialize before.
> 2018-01-05 08:44:30.359 ERROR 
> (OverseerStateUpdate-99296267947671552-127.0.1.1:8983_solr-n_00) [   
> ] o.a.s.c.Overseer Overseer could not process the current clusterstate state 
> update message, skipping the message: {
>   "core":"profidx",
>   "roles":null,
>   "base_url":"http://127.0.1.1:8983/solr;,
>   "node_name":"127.0.1.1:8983_solr",
>   "state":"down",
>   "shard":null,
>   "collection":"profidx",
>   "type":"NRT",
>   "operation":"state"}
> java.lang.NullPointerException
>   at org.apache.solr.cloud.Assign.defaultCounterValue(Assign.java:180)
>   at org.apache.solr.cloud.Assign.assignNode(Assign.java:119)
>   at 
> org.apache.solr.cloud.overseer.ReplicaMutator.updateState(ReplicaMutator.java:248)
>   at 
> org.apache.solr.cloud.overseer.ReplicaMutator.updateState(ReplicaMutator.java:232)
>   at 
> org.apache.solr.cloud.overseer.ReplicaMutator.setState(ReplicaMutator.java:205)
>   at 
> org.apache.solr.cloud.Overseer$ClusterStateUpdater.processMessage(Overseer.java:376)
>   at 
> org.apache.solr.cloud.Overseer$ClusterStateUpdater.processQueueItem(Overseer.java:273)
>   at 
> org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:224)
>   at java.base/java.lang.Thread.run(Thread.java:844)
> 2018-01-05 08:44:30.364 ERROR (qtp2009221452-21) [c:profidx   x:profidx] 
> o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Error 
> CREATEing SolrCore 'profidx': Unable to create core [profidx] Caused by: 
> Could not get shard id for core: profidx
>   at org.apache.solr.core.CoreContainer.create(CoreContainer.java:987)
>   at 
> org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91)
>   at 
> org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384)
>   at 
> org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389)
>   at 
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174)
>   at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
>   at 
> org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:735)
>   at 
> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:716)
>   at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:497)
>   at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
>   at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
>   at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1751)
>   at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
>   at 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
>   at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
>   at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
>   at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
>   at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
>   at 
> 

[JENKINS] Lucene-Solr-master-Windows (64bit/jdk-9.0.1) - Build # 7095 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/7095/
Java: 64bit/jdk-9.0.1 -XX:-UseCompressedOops -XX:+UseG1GC

3 tests failed.
FAILED:  
org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue.testDistributedQueueBlocking

Error Message:


Stack Trace:
java.util.concurrent.TimeoutException
at 
__randomizedtesting.SeedInfo.seed([FA509C7237C5A920:BFFAEE0B73971554]:0)
at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:204)
at 
org.apache.solr.cloud.autoscaling.sim.TestSimDistributedQueue.testDistributedQueueBlocking(TestSimDistributedQueue.java:116)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue

Error Message:
1 thread leaked from SUITE scope at 
org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue: 1) 
Thread[id=1878, name=sdqtest--429-thread-1, state=TIMED_WAITING, 

[jira] [Commented] (LUCENE-8099) Deprecate CustomScoreQuery, BoostedQuery and BoostingQuery

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16313443#comment-16313443
 ] 

David Smiley commented on LUCENE-8099:
--

With the addition of the nice new static utility methods on FunctionScoreQuery, 
I think you can replace the change you did to Solr's BoostQParserPlugin to 
simply be: {{return FunctionScoreQuery.boostByValue(input, 
vs.asDoubleValuesSource());}}  Maybe then inline it; I would but it's up to 
you.  Also the MIGRATE.txt notes should mention that these are replacements to 
some of the classes deleted here.

> Deprecate CustomScoreQuery, BoostedQuery and BoostingQuery
> --
>
> Key: LUCENE-8099
> URL: https://issues.apache.org/jira/browse/LUCENE-8099
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Minor
> Fix For: 7.3
>
> Attachments: LUCENE-8099-2.patch, LUCENE-8099.patch, LUCENE-8099.patch
>
>
> After LUCENE-7998, these three queries can all be replaced by a 
> FunctionScoreQuery.  Using lucene-expressions makes them much easier to use 
> as well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-EA] Lucene-Solr-master-Linux (64bit/jdk-10-ea+37) - Build # 21219 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/21219/
Java: 64bit/jdk-10-ea+37 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

3 tests failed.
FAILED:  org.apache.solr.cloud.ShardSplitTest.testSplitAfterFailedSplit

Error Message:
expected:<1> but was:<2>

Stack Trace:
java.lang.AssertionError: expected:<1> but was:<2>
at 
__randomizedtesting.SeedInfo.seed([5B3954BE0E4DFB88:A274C7113238B602]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at 
org.apache.solr.cloud.ShardSplitTest.testSplitAfterFailedSplit(ShardSplitTest.java:279)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[jira] [Updated] (LUCENE-8118) ArrayIndexOutOfBoundsException in TermsHashPerField.writeByte during indexing

[Robert Muir (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-8118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Muir updated LUCENE-8118:

Attachment: LUCENE-8118_test.patch

Here's a really bad test, but it works (takes about 2 minutes). 

lucene/core$ ant test -Dtestcase=TestIndexWriter 
-Dtestmethod=testAddDocumentsMassive -Dtests.heapsize=4G

{noformat}
  [junit4]  says مرحبا! Master seed: 1655BF16A8843A6A
   [junit4] Executing 1 suite with 1 JVM.
   [junit4] 
   [junit4] Started J0 PID(22813@localhost).
   [junit4] Suite: org.apache.lucene.index.TestIndexWriter
   [junit4] HEARTBEAT J0 PID(22813@localhost): 2018-01-05T11:27:48, stalled for 
71.2s at: TestIndexWriter.testAddDocumentsMassive
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestIndexWriter 
-Dtests.method=testAddDocumentsMassive -Dtests.seed=1655BF16A8843A6A 
-Dtests.locale=fr-FR -Dtests.timezone=Asia/Oral -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8
   [junit4] ERROR121s | TestIndexWriter.testAddDocumentsMassive <<<
   [junit4]> Throwable #1: java.lang.ArrayIndexOutOfBoundsException: -65536
   [junit4]>at 
__randomizedtesting.SeedInfo.seed([1655BF16A8843A6A:2B0B86082D338FEA]:0)
   [junit4]>at 
org.apache.lucene.index.TermsHashPerField.writeByte(TermsHashPerField.java:198)
   [junit4]>at 
org.apache.lucene.index.TermsHashPerField.writeVInt(TermsHashPerField.java:224)
   [junit4]>at 
org.apache.lucene.index.FreqProxTermsWriterPerField.writeProx(FreqProxTermsWriterPerField.java:80)
   [junit4]>at 
org.apache.lucene.index.FreqProxTermsWriterPerField.addTerm(FreqProxTermsWriterPerField.java:171)
   [junit4]>at 
org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:185)
   [junit4]>at 
org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:786)
   [junit4]>at 
org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:430)
   [junit4]>at 
org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:392)
   [junit4]>at 
org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:281)
   [junit4]>at 
org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:452)
   [junit4]>at 
org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1530)
   [junit4]>at 
org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1506)
   [junit4]>at 
org.apache.lucene.index.TestIndexWriter.testAddDocumentsMassive(TestIndexWriter.java:2994)
   [junit4]>at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: leaving temporary files on disk at: 
/home/rmuir/workspace/lucene-solr/lucene/build/core/test/J0/temp/lucene.index.TestIndexWriter_1655BF16A8843A6A-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70), 
sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@2a213e8b),
 locale=fr-FR, timezone=Asia/Oral
   [junit4]   2> NOTE: Linux 4.4.0-104-generic amd64/Oracle Corporation 
1.8.0_45 (64-bit)/cpus=8,threads=1,free=543468648,total=2733637632
   [junit4]   2> NOTE: All tests run in this JVM: [TestIndexWriter]
   [junit4] Completed [1/1 (1!)] in 121.55s, 1 test, 1 error <<< FAILURES!
   [junit4] 
   [junit4] 
   [junit4] Tests with failures [seed: 1655BF16A8843A6A]:
   [junit4]   - org.apache.lucene.index.TestIndexWriter.testAddDocumentsMassive
   [junit4] 
   [junit4] 
   [junit4] JVM J0: 0.38 ..   122.56 =   122.18s
   [junit4] Execution time total: 2 minutes 2 seconds
   [junit4] Tests summary: 1 suite, 1 test, 1 error

BUILD FAILED
/home/rmuir/workspace/lucene-solr/lucene/common-build.xml:1512: The following 
error occurred while executing this line:
/home/rmuir/workspace/lucene-solr/lucene/common-build.xml:1038: There were test 
failures: 1 suite, 1 test, 1 error [seed: 1655BF16A8843A6A]

Total time: 2 minutes 5 seconds
{noformat}

> ArrayIndexOutOfBoundsException in TermsHashPerField.writeByte during indexing
> -
>
> Key: LUCENE-8118
> URL: https://issues.apache.org/jira/browse/LUCENE-8118
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index
>Affects Versions: 7.2
> Environment: Debian/Stretch
> java version "1.8.0_144"  
>   
>Java(TM) SE Runtime 
> Environment (build 1.8.0_144-b01) 
>   
>Java 

Re: BugFix release 7.2.1

[Erick Erickson]

Neither of those Solr fixes are earth shatteringly important, they've both
been around for quite a while. I don't think it's urgent to include them.

That said, they're pretty simple and isolated so worth doing if Jim is
willing. But not worth straining much. I was just clearing out some backlog
over vacation.

Strictly up to you Jim.

Erick

On Fri, Jan 5, 2018 at 6:54 AM, David Smiley 
wrote:

> https://issues.apache.org/jira/browse/SOLR-11809 is in progress, should
> be easy and I think definitely worth backporting
>
> On Fri, Jan 5, 2018 at 8:52 AM Adrien Grand  wrote:
>
>> +1
>>
>> Looking at the changelog, 7.3 has 3 bug fixes for now: LUCENE-8077,
>> SOLR-11783 and SOLR-11555. The Lucene change doesn't seem worth
>> backporting, but maybe the Solr changes should?
>>
>> Le ven. 5 janv. 2018 à 12:40, jim ferenczi  a
>> écrit :
>>
>>> Hi,
>>> We discovered a bad bug in 7x that affects indices created in 6x with
>>> Lucene54DocValues format. The SortedNumericDocValues created with this
>>> format have a bug when advanceExact is used, the values retrieved for the
>>> docs when advanceExact returns true are invalid (the pointer to the values
>>> is not updated):
>>> https://issues.apache.org/jira/browse/LUCENE-8117
>>> This affects all indices created in 6x with sorted numeric doc values so
>>> I wanted to ask if anyone objects to a bugfix release for 7.2 (7.2.1). I
>>> also volunteer to be the release manager for this one if it is accepted.
>>>
>>> Jim
>>>
>>
>
> --
> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.
> solrenterprisesearchserver.com
>

[jira] [Commented] (LUCENE-8099) Deprecate CustomScoreQuery, BoostedQuery and BoostingQuery

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16313373#comment-16313373
 ] 

ASF subversion and git services commented on LUCENE-8099:
-

Commit e3c1947506b65bec93834e0607dc05a86537685e in lucene-solr's branch 
refs/heads/branch_7x from [~romseygeek]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=e3c1947 ]

LUCENE-8099: Add equality tests for FunctionScoreQuery


> Deprecate CustomScoreQuery, BoostedQuery and BoostingQuery
> --
>
> Key: LUCENE-8099
> URL: https://issues.apache.org/jira/browse/LUCENE-8099
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Minor
> Fix For: 7.3
>
> Attachments: LUCENE-8099-2.patch, LUCENE-8099.patch, LUCENE-8099.patch
>
>
> After LUCENE-7998, these three queries can all be replaced by a 
> FunctionScoreQuery.  Using lucene-expressions makes them much easier to use 
> as well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8099) Deprecate CustomScoreQuery, BoostedQuery and BoostingQuery

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16313374#comment-16313374
 ] 

ASF subversion and git services commented on LUCENE-8099:
-

Commit 3980aea18ded0a526900dd649fd5926ae65c3897 in lucene-solr's branch 
refs/heads/master from [~romseygeek]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=3980aea ]

LUCENE-8099: Add equality tests for FunctionScoreQuery


> Deprecate CustomScoreQuery, BoostedQuery and BoostingQuery
> --
>
> Key: LUCENE-8099
> URL: https://issues.apache.org/jira/browse/LUCENE-8099
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Minor
> Fix For: 7.3
>
> Attachments: LUCENE-8099-2.patch, LUCENE-8099.patch, LUCENE-8099.patch
>
>
> After LUCENE-7998, these three queries can all be replaced by a 
> FunctionScoreQuery.  Using lucene-expressions makes them much easier to use 
> as well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-8113) Allow terms dictionary lookups to be lazy when scores are not needed

[Alan Woodward (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-8113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Woodward resolved LUCENE-8113.
---
   Resolution: Fixed
 Assignee: Alan Woodward
Fix Version/s: master (8.0)

Committed in d250a1463d87380420afc90e381623c0dc470695 (I managed to mistype the 
issue number in the commit message)

> Allow terms dictionary lookups to be lazy when scores are not needed
> 
>
> Key: LUCENE-8113
> URL: https://issues.apache.org/jira/browse/LUCENE-8113
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Fix For: master (8.0)
>
> Attachments: LUCENE-8113-rename.patch, LUCENE-8113-rename.patch, 
> LUCENE-8113.patch, LUCENE-8113.patch, LUCENE-8113.patch
>
>
> LUCENE-7311 made it possible to avoid loading TermStates in cached 
> TermQueries.  It would be useful to extend this to other queries that use the 
> terms dictionary.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8118) ArrayIndexOutOfBoundsException in TermsHashPerField.writeByte during indexing

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16313337#comment-16313337
 ] 

Robert Muir commented on LUCENE-8118:
-

yeah, but we still need to fix the case where someone passes too many documents 
for addDocuments to succeed: it needs to be better than AIOOBE.

> ArrayIndexOutOfBoundsException in TermsHashPerField.writeByte during indexing
> -
>
> Key: LUCENE-8118
> URL: https://issues.apache.org/jira/browse/LUCENE-8118
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index
>Affects Versions: 7.2
> Environment: Debian/Stretch
> java version "1.8.0_144"  
>   
>Java(TM) SE Runtime 
> Environment (build 1.8.0_144-b01) 
>   
>Java HotSpot(TM) 64-Bit Server VM (build 
> 25.144-b01, mixed mode)
>Reporter: Laura Dietz
>
> Indexing a large collection of about 20 million paragraph-sized documents 
> results in an ArrayIndexOutOfBoundsException in 
> org.apache.lucene.index.TermsHashPerField.writeByte  (full stack trace 
> below). 
> The bug is possibly related to issues described in 
> [here|http://lucene.472066.n3.nabble.com/ArrayIndexOutOfBoundsException-65536-td3661945.html]
>   and [SOLR-10936|https://issues.apache.org/jira/browse/SOLR-10936] -- but I 
> am not using SOLR, I am directly using Lucene Core.
> The issue can be reproduced using code from  [GitHub 
> trec-car-tools-example|https://github.com/TREMA-UNH/trec-car-tools/tree/lucene-bug/trec-car-tools-example]
>  
> - compile with `mvn compile assembly:single`
> - run with `java -cp 
> ./target/treccar-tools-example-0.1-jar-with-dependencies.jar 
> edu.unh.cs.TrecCarBuildLuceneIndex paragraphs paragraphCorpus.cbor indexDir`
> Where paragraphCorpus.cbor is contained in this 
> [archive|http://trec-car.cs.unh.edu/datareleases/v2.0-snapshot/archive-paragraphCorpus.tar.xz]
> Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -65536   
> at 
> org.apache.lucene.index.TermsHashPerField.writeByte(TermsHashPerField.java:198)
>   
>at 
> org.apache.lucene.index.TermsHashPerField.writeVInt(TermsHashPerField.java:224)
>   
>at 
> org.apache.lucene.index.FreqProxTermsWriterPerField.addTerm(FreqProxTermsWriterPerField.java:159)
>   
>  at 
> org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:185) 
>   
> at 
> org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:786)
>   
>at 
> org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:430)
>   
>   at 
> org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:392)
>   
>at 
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:281)
>   
>at 
> org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:451)
>   
>  at 
> org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1532)
>   
> at 
> org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1508)
> at 
> edu.unh.cs.TrecCarBuildLuceneIndex.main(TrecCarBuildLuceneIndex.java:55)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To 

[jira] [Commented] (LUCENE-8099) Deprecate CustomScoreQuery, BoostedQuery and BoostingQuery

[Alan Woodward (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16313328#comment-16313328
 ] 

Alan Woodward commented on LUCENE-8099:
---

What pushed me over the line on the static methods was the realisation that my 
suggested replacement for BoostingQuery didn't do what I thought it was doing, 
although Hoss's -1 helped :) . And I think I also misread your earlier comment 
about static methods, thinking that you meant them for creating value sources, 
rather than as query replacements.

There isn't an issue with equality comparisons here, because the static methods 
use concrete classes with .equals() and .hashCode() implemented on them for 
their sources.  They should have tests for this though, will add them in.

> Deprecate CustomScoreQuery, BoostedQuery and BoostingQuery
> --
>
> Key: LUCENE-8099
> URL: https://issues.apache.org/jira/browse/LUCENE-8099
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Minor
> Fix For: 7.3
>
> Attachments: LUCENE-8099-2.patch, LUCENE-8099.patch, LUCENE-8099.patch
>
>
> After LUCENE-7998, these three queries can all be replaced by a 
> FunctionScoreQuery.  Using lucene-expressions makes them much easier to use 
> as well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8118) ArrayIndexOutOfBoundsException in TermsHashPerField.writeByte during indexing

[Laura Dietz (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16313323#comment-16313323
 ] 

Laura Dietz commented on LUCENE-8118:
-

I think my mistake was to abuse addDocuments(iterator).

I switched to addDocument(doc) with a commit every so often (see master branch)


> ArrayIndexOutOfBoundsException in TermsHashPerField.writeByte during indexing
> -
>
> Key: LUCENE-8118
> URL: https://issues.apache.org/jira/browse/LUCENE-8118
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index
>Affects Versions: 7.2
> Environment: Debian/Stretch
> java version "1.8.0_144"  
>   
>Java(TM) SE Runtime 
> Environment (build 1.8.0_144-b01) 
>   
>Java HotSpot(TM) 64-Bit Server VM (build 
> 25.144-b01, mixed mode)
>Reporter: Laura Dietz
>
> Indexing a large collection of about 20 million paragraph-sized documents 
> results in an ArrayIndexOutOfBoundsException in 
> org.apache.lucene.index.TermsHashPerField.writeByte  (full stack trace 
> below). 
> The bug is possibly related to issues described in 
> [here|http://lucene.472066.n3.nabble.com/ArrayIndexOutOfBoundsException-65536-td3661945.html]
>   and [SOLR-10936|https://issues.apache.org/jira/browse/SOLR-10936] -- but I 
> am not using SOLR, I am directly using Lucene Core.
> The issue can be reproduced using code from  [GitHub 
> trec-car-tools-example|https://github.com/TREMA-UNH/trec-car-tools/tree/lucene-bug/trec-car-tools-example]
>  
> - compile with `mvn compile assembly:single`
> - run with `java -cp 
> ./target/treccar-tools-example-0.1-jar-with-dependencies.jar 
> edu.unh.cs.TrecCarBuildLuceneIndex paragraphs paragraphCorpus.cbor indexDir`
> Where paragraphCorpus.cbor is contained in this 
> [archive|http://trec-car.cs.unh.edu/datareleases/v2.0-snapshot/archive-paragraphCorpus.tar.xz]
> Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -65536   
> at 
> org.apache.lucene.index.TermsHashPerField.writeByte(TermsHashPerField.java:198)
>   
>at 
> org.apache.lucene.index.TermsHashPerField.writeVInt(TermsHashPerField.java:224)
>   
>at 
> org.apache.lucene.index.FreqProxTermsWriterPerField.addTerm(FreqProxTermsWriterPerField.java:159)
>   
>  at 
> org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:185) 
>   
> at 
> org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:786)
>   
>at 
> org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:430)
>   
>   at 
> org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:392)
>   
>at 
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:281)
>   
>at 
> org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:451)
>   
>  at 
> org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1532)
>   
> at 
> org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1508)
> at 
> edu.unh.cs.TrecCarBuildLuceneIndex.main(TrecCarBuildLuceneIndex.java:55)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To 

[jira] [Commented] (LUCENE-8118) ArrayIndexOutOfBoundsException in TermsHashPerField.writeByte during indexing

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16313322#comment-16313322
 ] 

Robert Muir commented on LUCENE-8118:
-

whatever we decide to do, we can be sure that AIOOBE is not the right answer :)

> ArrayIndexOutOfBoundsException in TermsHashPerField.writeByte during indexing
> -
>
> Key: LUCENE-8118
> URL: https://issues.apache.org/jira/browse/LUCENE-8118
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index
>Affects Versions: 7.2
> Environment: Debian/Stretch
> java version "1.8.0_144"  
>   
>Java(TM) SE Runtime 
> Environment (build 1.8.0_144-b01) 
>   
>Java HotSpot(TM) 64-Bit Server VM (build 
> 25.144-b01, mixed mode)
>Reporter: Laura Dietz
>
> Indexing a large collection of about 20 million paragraph-sized documents 
> results in an ArrayIndexOutOfBoundsException in 
> org.apache.lucene.index.TermsHashPerField.writeByte  (full stack trace 
> below). 
> The bug is possibly related to issues described in 
> [here|http://lucene.472066.n3.nabble.com/ArrayIndexOutOfBoundsException-65536-td3661945.html]
>   and [SOLR-10936|https://issues.apache.org/jira/browse/SOLR-10936] -- but I 
> am not using SOLR, I am directly using Lucene Core.
> The issue can be reproduced using code from  [GitHub 
> trec-car-tools-example|https://github.com/TREMA-UNH/trec-car-tools/tree/lucene-bug/trec-car-tools-example]
>  
> - compile with `mvn compile assembly:single`
> - run with `java -cp 
> ./target/treccar-tools-example-0.1-jar-with-dependencies.jar 
> edu.unh.cs.TrecCarBuildLuceneIndex paragraphs paragraphCorpus.cbor indexDir`
> Where paragraphCorpus.cbor is contained in this 
> [archive|http://trec-car.cs.unh.edu/datareleases/v2.0-snapshot/archive-paragraphCorpus.tar.xz]
> Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -65536   
> at 
> org.apache.lucene.index.TermsHashPerField.writeByte(TermsHashPerField.java:198)
>   
>at 
> org.apache.lucene.index.TermsHashPerField.writeVInt(TermsHashPerField.java:224)
>   
>at 
> org.apache.lucene.index.FreqProxTermsWriterPerField.addTerm(FreqProxTermsWriterPerField.java:159)
>   
>  at 
> org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:185) 
>   
> at 
> org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:786)
>   
>at 
> org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:430)
>   
>   at 
> org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:392)
>   
>at 
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:281)
>   
>at 
> org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:451)
>   
>  at 
> org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1532)
>   
> at 
> org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1508)
> at 
> edu.unh.cs.TrecCarBuildLuceneIndex.main(TrecCarBuildLuceneIndex.java:55)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For 

[jira] [Commented] (LUCENE-8118) ArrayIndexOutOfBoundsException in TermsHashPerField.writeByte during indexing

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16313318#comment-16313318
 ] 

Robert Muir commented on LUCENE-8118:
-

Well, I understand the bug, but not sure what the fix is.

Indexing code implements Iterable etc to pull in the docs, and makes one single 
call to addDocuments().

This is supposed to be an "atomic add" of multiple documents at once which 
gives certain guarantees: needed for nested documents and features like that so 
they document IDs will be aligned in a particular way.

In your case, its too much data, IndexWriter isn't going to be able to do 200M 
docs in one operation like this.

> ArrayIndexOutOfBoundsException in TermsHashPerField.writeByte during indexing
> -
>
> Key: LUCENE-8118
> URL: https://issues.apache.org/jira/browse/LUCENE-8118
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index
>Affects Versions: 7.2
> Environment: Debian/Stretch
> java version "1.8.0_144"  
>   
>Java(TM) SE Runtime 
> Environment (build 1.8.0_144-b01) 
>   
>Java HotSpot(TM) 64-Bit Server VM (build 
> 25.144-b01, mixed mode)
>Reporter: Laura Dietz
>
> Indexing a large collection of about 20 million paragraph-sized documents 
> results in an ArrayIndexOutOfBoundsException in 
> org.apache.lucene.index.TermsHashPerField.writeByte  (full stack trace 
> below). 
> The bug is possibly related to issues described in 
> [here|http://lucene.472066.n3.nabble.com/ArrayIndexOutOfBoundsException-65536-td3661945.html]
>   and [SOLR-10936|https://issues.apache.org/jira/browse/SOLR-10936] -- but I 
> am not using SOLR, I am directly using Lucene Core.
> The issue can be reproduced using code from  [GitHub 
> trec-car-tools-example|https://github.com/TREMA-UNH/trec-car-tools/tree/lucene-bug/trec-car-tools-example]
>  
> - compile with `mvn compile assembly:single`
> - run with `java -cp 
> ./target/treccar-tools-example-0.1-jar-with-dependencies.jar 
> edu.unh.cs.TrecCarBuildLuceneIndex paragraphs paragraphCorpus.cbor indexDir`
> Where paragraphCorpus.cbor is contained in this 
> [archive|http://trec-car.cs.unh.edu/datareleases/v2.0-snapshot/archive-paragraphCorpus.tar.xz]
> Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -65536   
> at 
> org.apache.lucene.index.TermsHashPerField.writeByte(TermsHashPerField.java:198)
>   
>at 
> org.apache.lucene.index.TermsHashPerField.writeVInt(TermsHashPerField.java:224)
>   
>at 
> org.apache.lucene.index.FreqProxTermsWriterPerField.addTerm(FreqProxTermsWriterPerField.java:159)
>   
>  at 
> org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:185) 
>   
> at 
> org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:786)
>   
>at 
> org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:430)
>   
>   at 
> org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:392)
>   
>at 
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:281)
>   
>at 
> org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:451)
>   
>  at 
> org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1532)
>   
>

[JENKINS] Lucene-Solr-7.x-Linux (64bit/jdk-9.0.1) - Build # 1124 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/1124/
Java: 64bit/jdk-9.0.1 -XX:+UseCompressedOops -XX:+UseSerialGC

4 tests failed.
FAILED:  
org.apache.solr.client.solrj.io.stream.StreamExpressionTest.testUpdateStream

Error Message:
Error from server at http://127.0.0.1:40679/solr: create the collection time 
out:180s

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:40679/solr: create the collection time out:180s
at 
__randomizedtesting.SeedInfo.seed([73FACDF89453EC37:BBBD09829D5B1167]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at 
org.apache.solr.client.solrj.io.stream.StreamExpressionTest.testUpdateStream(StreamExpressionTest.java:3958)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[jira] [Commented] (LUCENE-8118) ArrayIndexOutOfBoundsException in TermsHashPerField.writeByte during indexing

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16313299#comment-16313299
 ] 

Robert Muir commented on LUCENE-8118:
-

It is nothing like that, it is simply a bug.

> ArrayIndexOutOfBoundsException in TermsHashPerField.writeByte during indexing
> -
>
> Key: LUCENE-8118
> URL: https://issues.apache.org/jira/browse/LUCENE-8118
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index
>Affects Versions: 7.2
> Environment: Debian/Stretch
> java version "1.8.0_144"  
>   
>Java(TM) SE Runtime 
> Environment (build 1.8.0_144-b01) 
>   
>Java HotSpot(TM) 64-Bit Server VM (build 
> 25.144-b01, mixed mode)
>Reporter: Laura Dietz
>
> Indexing a large collection of about 20 million paragraph-sized documents 
> results in an ArrayIndexOutOfBoundsException in 
> org.apache.lucene.index.TermsHashPerField.writeByte  (full stack trace 
> below). 
> The bug is possibly related to issues described in 
> [here|http://lucene.472066.n3.nabble.com/ArrayIndexOutOfBoundsException-65536-td3661945.html]
>   and [SOLR-10936|https://issues.apache.org/jira/browse/SOLR-10936] -- but I 
> am not using SOLR, I am directly using Lucene Core.
> The issue can be reproduced using code from  [GitHub 
> trec-car-tools-example|https://github.com/TREMA-UNH/trec-car-tools/tree/lucene-bug/trec-car-tools-example]
>  
> - compile with `mvn compile assembly:single`
> - run with `java -cp 
> ./target/treccar-tools-example-0.1-jar-with-dependencies.jar 
> edu.unh.cs.TrecCarBuildLuceneIndex paragraphs paragraphCorpus.cbor indexDir`
> Where paragraphCorpus.cbor is contained in this 
> [archive|http://trec-car.cs.unh.edu/datareleases/v2.0-snapshot/archive-paragraphCorpus.tar.xz]
> Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -65536   
> at 
> org.apache.lucene.index.TermsHashPerField.writeByte(TermsHashPerField.java:198)
>   
>at 
> org.apache.lucene.index.TermsHashPerField.writeVInt(TermsHashPerField.java:224)
>   
>at 
> org.apache.lucene.index.FreqProxTermsWriterPerField.addTerm(FreqProxTermsWriterPerField.java:159)
>   
>  at 
> org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:185) 
>   
> at 
> org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:786)
>   
>at 
> org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:430)
>   
>   at 
> org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:392)
>   
>at 
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:281)
>   
>at 
> org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:451)
>   
>  at 
> org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1532)
>   
> at 
> org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1508)
> at 
> edu.unh.cs.TrecCarBuildLuceneIndex.main(TrecCarBuildLuceneIndex.java:55)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: 

[jira] [Commented] (LUCENE-8118) ArrayIndexOutOfBoundsException in TermsHashPerField.writeByte during indexing

[Laura Dietz (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16313292#comment-16313292
 ] 

Laura Dietz commented on LUCENE-8118:
-

Robert, that would be even better!

It is difficult to guess what the right interval of issuing a commits is. I 
understand that some hand tuning might be necessary to get the highest 
performance for given resource constraints. If the issue is a buffer that is 
filling up, it would be helpful to have some form of an emergency auto-commit.

> ArrayIndexOutOfBoundsException in TermsHashPerField.writeByte during indexing
> -
>
> Key: LUCENE-8118
> URL: https://issues.apache.org/jira/browse/LUCENE-8118
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index
>Affects Versions: 7.2
> Environment: Debian/Stretch
> java version "1.8.0_144"  
>   
>Java(TM) SE Runtime 
> Environment (build 1.8.0_144-b01) 
>   
>Java HotSpot(TM) 64-Bit Server VM (build 
> 25.144-b01, mixed mode)
>Reporter: Laura Dietz
>
> Indexing a large collection of about 20 million paragraph-sized documents 
> results in an ArrayIndexOutOfBoundsException in 
> org.apache.lucene.index.TermsHashPerField.writeByte  (full stack trace 
> below). 
> The bug is possibly related to issues described in 
> [here|http://lucene.472066.n3.nabble.com/ArrayIndexOutOfBoundsException-65536-td3661945.html]
>   and [SOLR-10936|https://issues.apache.org/jira/browse/SOLR-10936] -- but I 
> am not using SOLR, I am directly using Lucene Core.
> The issue can be reproduced using code from  [GitHub 
> trec-car-tools-example|https://github.com/TREMA-UNH/trec-car-tools/tree/lucene-bug/trec-car-tools-example]
>  
> - compile with `mvn compile assembly:single`
> - run with `java -cp 
> ./target/treccar-tools-example-0.1-jar-with-dependencies.jar 
> edu.unh.cs.TrecCarBuildLuceneIndex paragraphs paragraphCorpus.cbor indexDir`
> Where paragraphCorpus.cbor is contained in this 
> [archive|http://trec-car.cs.unh.edu/datareleases/v2.0-snapshot/archive-paragraphCorpus.tar.xz]
> Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -65536   
> at 
> org.apache.lucene.index.TermsHashPerField.writeByte(TermsHashPerField.java:198)
>   
>at 
> org.apache.lucene.index.TermsHashPerField.writeVInt(TermsHashPerField.java:224)
>   
>at 
> org.apache.lucene.index.FreqProxTermsWriterPerField.addTerm(FreqProxTermsWriterPerField.java:159)
>   
>  at 
> org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:185) 
>   
> at 
> org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:786)
>   
>at 
> org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:430)
>   
>   at 
> org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:392)
>   
>at 
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:281)
>   
>at 
> org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:451)
>   
>  at 
> org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1532)
>   
> at 
> org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1508)
> at 
> 

[jira] [Commented] (LUCENE-8119) Remove SimScorer.maxScore(maxFreq)

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16313287#comment-16313287
 ] 

Robert Muir commented on LUCENE-8119:
-

I think we can see it does what we want via the assert: 

{code}
assert state.getLength() > 0;
{code}

https://github.com/apache/lucene-solr/blob/master/lucene/test-framework/src/java/org/apache/lucene/search/similarities/AssertingSimilarity.java#L37

So it would be good to simplify the language, keeping in mind this method will 
never be called for empty fields.


> Remove SimScorer.maxScore(maxFreq)
> --
>
> Key: LUCENE-8119
> URL: https://issues.apache.org/jira/browse/LUCENE-8119
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Adrien Grand
>Priority: Minor
> Attachments: LUCENE-8119.patch
>
>
> Now that SimScorer directly takes a frequency and a norm, this can be 
> replaced with {{SimScorer.freq(maxFreq, 1)}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8118) ArrayIndexOutOfBoundsException in TermsHashPerField.writeByte during indexing

[Diego Ceccarelli (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16313284#comment-16313284
 ] 

Diego Ceccarelli commented on LUCENE-8118:
--

I agree, that was just a workaround for [~laura-dietz] :) 

> ArrayIndexOutOfBoundsException in TermsHashPerField.writeByte during indexing
> -
>
> Key: LUCENE-8118
> URL: https://issues.apache.org/jira/browse/LUCENE-8118
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index
>Affects Versions: 7.2
> Environment: Debian/Stretch
> java version "1.8.0_144"  
>   
>Java(TM) SE Runtime 
> Environment (build 1.8.0_144-b01) 
>   
>Java HotSpot(TM) 64-Bit Server VM (build 
> 25.144-b01, mixed mode)
>Reporter: Laura Dietz
>
> Indexing a large collection of about 20 million paragraph-sized documents 
> results in an ArrayIndexOutOfBoundsException in 
> org.apache.lucene.index.TermsHashPerField.writeByte  (full stack trace 
> below). 
> The bug is possibly related to issues described in 
> [here|http://lucene.472066.n3.nabble.com/ArrayIndexOutOfBoundsException-65536-td3661945.html]
>   and [SOLR-10936|https://issues.apache.org/jira/browse/SOLR-10936] -- but I 
> am not using SOLR, I am directly using Lucene Core.
> The issue can be reproduced using code from  [GitHub 
> trec-car-tools-example|https://github.com/TREMA-UNH/trec-car-tools/tree/lucene-bug/trec-car-tools-example]
>  
> - compile with `mvn compile assembly:single`
> - run with `java -cp 
> ./target/treccar-tools-example-0.1-jar-with-dependencies.jar 
> edu.unh.cs.TrecCarBuildLuceneIndex paragraphs paragraphCorpus.cbor indexDir`
> Where paragraphCorpus.cbor is contained in this 
> [archive|http://trec-car.cs.unh.edu/datareleases/v2.0-snapshot/archive-paragraphCorpus.tar.xz]
> Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -65536   
> at 
> org.apache.lucene.index.TermsHashPerField.writeByte(TermsHashPerField.java:198)
>   
>at 
> org.apache.lucene.index.TermsHashPerField.writeVInt(TermsHashPerField.java:224)
>   
>at 
> org.apache.lucene.index.FreqProxTermsWriterPerField.addTerm(FreqProxTermsWriterPerField.java:159)
>   
>  at 
> org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:185) 
>   
> at 
> org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:786)
>   
>at 
> org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:430)
>   
>   at 
> org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:392)
>   
>at 
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:281)
>   
>at 
> org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:451)
>   
>  at 
> org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1532)
>   
> at 
> org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1508)
> at 
> edu.unh.cs.TrecCarBuildLuceneIndex.main(TrecCarBuildLuceneIndex.java:55)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional 

[jira] [Commented] (LUCENE-8119) Remove SimScorer.maxScore(maxFreq)

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16313280#comment-16313280
 ] 

Robert Muir commented on LUCENE-8119:
-

Right, I think its better to look at 0 as just an impossible/unused value: 
that's the perspective at query-time for the similarity class so it would be 
good to match that at index-time. All the divide by zero possibilities caused a 
real mess before.

> Remove SimScorer.maxScore(maxFreq)
> --
>
> Key: LUCENE-8119
> URL: https://issues.apache.org/jira/browse/LUCENE-8119
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Adrien Grand
>Priority: Minor
> Attachments: LUCENE-8119.patch
>
>
> Now that SimScorer directly takes a frequency and a norm, this can be 
> replaced with {{SimScorer.freq(maxFreq, 1)}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-7.x-Solaris (64bit/jdk1.8.0) - Build # 375 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Solaris/375/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC

3 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation

Error Message:
2 threads leaked from SUITE scope at 
org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation: 1) 
Thread[id=37645, name=jetty-launcher-8147-thread-2-EventThread, state=WAITING, 
group=TGRP-TestSolrCloudWithSecureImpersonation] at 
sun.misc.Unsafe.park(Native Method) at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
 at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) 
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)
2) Thread[id=37644, 
name=jetty-launcher-8147-thread-2-SendThread(127.0.0.1:59478), 
state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] 
at java.lang.Thread.sleep(Native Method) at 
org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
 at 
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)
 at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE 
scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation: 
   1) Thread[id=37645, name=jetty-launcher-8147-thread-2-EventThread, 
state=WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)
   2) Thread[id=37644, 
name=jetty-launcher-8147-thread-2-SendThread(127.0.0.1:59478), 
state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at java.lang.Thread.sleep(Native Method)
at 
org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
at 
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)
at __randomizedtesting.SeedInfo.seed([A5B9F9685486]:0)


FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation

Error Message:
There are still zombie threads that couldn't be terminated:1) 
Thread[id=37644, name=jetty-launcher-8147-thread-2-SendThread(127.0.0.1:59478), 
state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] 
at java.lang.Thread.sleep(Native Method) at 
org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
 at 
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)
 at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie 
threads that couldn't be terminated:
   1) Thread[id=37644, 
name=jetty-launcher-8147-thread-2-SendThread(127.0.0.1:59478), 
state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at java.lang.Thread.sleep(Native Method)
at 
org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
at 
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)
at __randomizedtesting.SeedInfo.seed([A5B9F9685486]:0)


FAILED:  
org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeLostTriggerRestoreState

Error Message:
The trigger did not fire at all

Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at 
__randomizedtesting.SeedInfo.seed([A5B9F9685486:8E462779B3104156]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeLostTriggerRestoreState(TestTriggerIntegration.java:307)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 

[jira] [Commented] (LUCENE-8119) Remove SimScorer.maxScore(maxFreq)

[Adrien Grand (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16313272#comment-16313272
 ] 

Adrien Grand commented on LUCENE-8119:
--

I overspecified it, but my intention was to make sure that non-empty fields 
never get 0 as a norm value, so that 1 is the norm value that produces the best 
scores for a given freq. I agree with you there is no reason to enforce that it 
is 0 on empty fields.

> Remove SimScorer.maxScore(maxFreq)
> --
>
> Key: LUCENE-8119
> URL: https://issues.apache.org/jira/browse/LUCENE-8119
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Adrien Grand
>Priority: Minor
> Attachments: LUCENE-8119.patch
>
>
> Now that SimScorer directly takes a frequency and a norm, this can be 
> replaced with {{SimScorer.freq(maxFreq, 1)}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8118) ArrayIndexOutOfBoundsException in TermsHashPerField.writeByte during indexing

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16313271#comment-16313271
 ] 

Robert Muir commented on LUCENE-8118:
-

Issuing unnecessary commits is just masking the issue: you shouldn't see this 
exception.

> ArrayIndexOutOfBoundsException in TermsHashPerField.writeByte during indexing
> -
>
> Key: LUCENE-8118
> URL: https://issues.apache.org/jira/browse/LUCENE-8118
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index
>Affects Versions: 7.2
> Environment: Debian/Stretch
> java version "1.8.0_144"  
>   
>Java(TM) SE Runtime 
> Environment (build 1.8.0_144-b01) 
>   
>Java HotSpot(TM) 64-Bit Server VM (build 
> 25.144-b01, mixed mode)
>Reporter: Laura Dietz
>
> Indexing a large collection of about 20 million paragraph-sized documents 
> results in an ArrayIndexOutOfBoundsException in 
> org.apache.lucene.index.TermsHashPerField.writeByte  (full stack trace 
> below). 
> The bug is possibly related to issues described in 
> [here|http://lucene.472066.n3.nabble.com/ArrayIndexOutOfBoundsException-65536-td3661945.html]
>   and [SOLR-10936|https://issues.apache.org/jira/browse/SOLR-10936] -- but I 
> am not using SOLR, I am directly using Lucene Core.
> The issue can be reproduced using code from  [GitHub 
> trec-car-tools-example|https://github.com/TREMA-UNH/trec-car-tools/tree/lucene-bug/trec-car-tools-example]
>  
> - compile with `mvn compile assembly:single`
> - run with `java -cp 
> ./target/treccar-tools-example-0.1-jar-with-dependencies.jar 
> edu.unh.cs.TrecCarBuildLuceneIndex paragraphs paragraphCorpus.cbor indexDir`
> Where paragraphCorpus.cbor is contained in this 
> [archive|http://trec-car.cs.unh.edu/datareleases/v2.0-snapshot/archive-paragraphCorpus.tar.xz]
> Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -65536   
> at 
> org.apache.lucene.index.TermsHashPerField.writeByte(TermsHashPerField.java:198)
>   
>at 
> org.apache.lucene.index.TermsHashPerField.writeVInt(TermsHashPerField.java:224)
>   
>at 
> org.apache.lucene.index.FreqProxTermsWriterPerField.addTerm(FreqProxTermsWriterPerField.java:159)
>   
>  at 
> org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:185) 
>   
> at 
> org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:786)
>   
>at 
> org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:430)
>   
>   at 
> org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:392)
>   
>at 
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:281)
>   
>at 
> org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:451)
>   
>  at 
> org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1532)
>   
> at 
> org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1508)
> at 
> edu.unh.cs.TrecCarBuildLuceneIndex.main(TrecCarBuildLuceneIndex.java:55)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org

[jira] [Commented] (LUCENE-8118) ArrayIndexOutOfBoundsException in TermsHashPerField.writeByte during indexing

[Laura Dietz (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16313256#comment-16313256
 ] 

Laura Dietz commented on LUCENE-8118:
-

Yes, that works - Thanks, Diego!

I think I could have been helped with an Exception message that indicates 
"Buffer full, call index.commit!"




> ArrayIndexOutOfBoundsException in TermsHashPerField.writeByte during indexing
> -
>
> Key: LUCENE-8118
> URL: https://issues.apache.org/jira/browse/LUCENE-8118
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index
>Affects Versions: 7.2
> Environment: Debian/Stretch
> java version "1.8.0_144"  
>   
>Java(TM) SE Runtime 
> Environment (build 1.8.0_144-b01) 
>   
>Java HotSpot(TM) 64-Bit Server VM (build 
> 25.144-b01, mixed mode)
>Reporter: Laura Dietz
>
> Indexing a large collection of about 20 million paragraph-sized documents 
> results in an ArrayIndexOutOfBoundsException in 
> org.apache.lucene.index.TermsHashPerField.writeByte  (full stack trace 
> below). 
> The bug is possibly related to issues described in 
> [here|http://lucene.472066.n3.nabble.com/ArrayIndexOutOfBoundsException-65536-td3661945.html]
>   and [SOLR-10936|https://issues.apache.org/jira/browse/SOLR-10936] -- but I 
> am not using SOLR, I am directly using Lucene Core.
> The issue can be reproduced using code from  [GitHub 
> trec-car-tools-example|https://github.com/TREMA-UNH/trec-car-tools/tree/lucene-bug/trec-car-tools-example]
>  
> - compile with `mvn compile assembly:single`
> - run with `java -cp 
> ./target/treccar-tools-example-0.1-jar-with-dependencies.jar 
> edu.unh.cs.TrecCarBuildLuceneIndex paragraphs paragraphCorpus.cbor indexDir`
> Where paragraphCorpus.cbor is contained in this 
> [archive|http://trec-car.cs.unh.edu/datareleases/v2.0-snapshot/archive-paragraphCorpus.tar.xz]
> Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -65536   
> at 
> org.apache.lucene.index.TermsHashPerField.writeByte(TermsHashPerField.java:198)
>   
>at 
> org.apache.lucene.index.TermsHashPerField.writeVInt(TermsHashPerField.java:224)
>   
>at 
> org.apache.lucene.index.FreqProxTermsWriterPerField.addTerm(FreqProxTermsWriterPerField.java:159)
>   
>  at 
> org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:185) 
>   
> at 
> org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:786)
>   
>at 
> org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:430)
>   
>   at 
> org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:392)
>   
>at 
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:281)
>   
>at 
> org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:451)
>   
>  at 
> org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1532)
>   
> at 
> org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1508)
> at 
> edu.unh.cs.TrecCarBuildLuceneIndex.main(TrecCarBuildLuceneIndex.java:55)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To 

Re: BugFix release 7.2.1

[David Smiley]

https://issues.apache.org/jira/browse/SOLR-11809 is in progress, should be
easy and I think definitely worth backporting

On Fri, Jan 5, 2018 at 8:52 AM Adrien Grand  wrote:

> +1
>
> Looking at the changelog, 7.3 has 3 bug fixes for now: LUCENE-8077,
> SOLR-11783 and SOLR-11555. The Lucene change doesn't seem worth
> backporting, but maybe the Solr changes should?
>
> Le ven. 5 janv. 2018 à 12:40, jim ferenczi  a
> écrit :
>
>> Hi,
>> We discovered a bad bug in 7x that affects indices created in 6x with
>> Lucene54DocValues format. The SortedNumericDocValues created with this
>> format have a bug when advanceExact is used, the values retrieved for the
>> docs when advanceExact returns true are invalid (the pointer to the values
>> is not updated):
>> https://issues.apache.org/jira/browse/LUCENE-8117
>> This affects all indices created in 6x with sorted numeric doc values so
>> I wanted to ask if anyone objects to a bugfix release for 7.2 (7.2.1). I
>> also volunteer to be the release manager for this one if it is accepted.
>>
>> Jim
>>
>

-- 
Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
http://www.solrenterprisesearchserver.com

[jira] [Commented] (LUCENE-8099) Deprecate CustomScoreQuery, BoostedQuery and BoostingQuery

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16313220#comment-16313220
 ] 

David Smiley commented on LUCENE-8099:
--

The two additional convenience methods with backing value sources are fantastic 
(thank you) -- it addresses my earlier concerns.   it seems lightweight and 
needn't involve another module (expressions).  At the time I said this, you 
responded to say this:

bq. DoubleValuesSource had a helper to create sources that used arbitrary 
functions when I first committed it, but I removed it a bit later because it 
messed up equality tests (essentially, you can't check Java closures for 
equality).

Your latest commit doesn't show any changes to equality tests... are we talking 
about different things?  Was Hoss's "-1" what pushed you over to make these 
changes?  I'm wondering if I should give -1's more... I think I felt about as 
strongly as Hoss but didn't express it.

> Deprecate CustomScoreQuery, BoostedQuery and BoostingQuery
> --
>
> Key: LUCENE-8099
> URL: https://issues.apache.org/jira/browse/LUCENE-8099
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Minor
> Fix For: 7.3
>
> Attachments: LUCENE-8099-2.patch, LUCENE-8099.patch, LUCENE-8099.patch
>
>
> After LUCENE-7998, these three queries can all be replaced by a 
> FunctionScoreQuery.  Using lucene-expressions makes them much easier to use 
> as well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (LUCENE-2287) Unexpected terms are highlighted within nested SpanQuery instances

[David Smiley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-2287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Smiley reassigned LUCENE-2287:


 Assignee: David Smiley
Fix Version/s: 7.3

> Unexpected terms are highlighted within nested SpanQuery instances
> --
>
> Key: LUCENE-2287
> URL: https://issues.apache.org/jira/browse/LUCENE-2287
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/highlighter
>Affects Versions: 2.9.1
> Environment: Linux, Solaris, Windows
>Reporter: Michael Goddard
>Assignee: David Smiley
>Priority: Minor
> Fix For: 7.3
>
> Attachments: LUCENE-2287.patch, LUCENE-2287.patch, LUCENE-2287.patch, 
> LUCENE-2287.patch, LUCENE-2287.patch, LUCENE-2287.patch
>
>   Original Estimate: 336h
>  Remaining Estimate: 336h
>
> I haven't yet been able to resolve why I'm seeing spurious highlighting in 
> nested SpanQuery instances.  Briefly, the issue is illustrated by the second 
> instance of "Lucene" being highlighted in the test below, when it doesn't 
> satisfy the inner span.  There's been some discussion about this on the 
> java-dev list, and I'm opening this issue now because I have made some 
> initial progress on this.
> This new test, added to the  HighlighterTest class in lucene_2_9_1, 
> illustrates this:
> /*
>  * Ref: http://www.lucidimagination.com/blog/2009/07/18/the-spanquery/
>  */
> public void testHighlightingNestedSpans2() throws Exception {
>   String theText = "The Lucene was made by Doug Cutting and Lucene great 
> Hadoop was"; // Problem
>   //String theText = "The Lucene was made by Doug Cutting and the great 
> Hadoop was"; // Works okay
>   String fieldName = "SOME_FIELD_NAME";
>   SpanNearQuery spanNear = new SpanNearQuery(new SpanQuery[] {
> new SpanTermQuery(new Term(fieldName, "lucene")),
> new SpanTermQuery(new Term(fieldName, "doug")) }, 5, true);
>   Query query = new SpanNearQuery(new SpanQuery[] { spanNear,
> new SpanTermQuery(new Term(fieldName, "hadoop")) }, 4, true);
>   String expected = "The Lucene was made by Doug Cutting and 
> Lucene great Hadoop was";
>   //String expected = "The Lucene was made by Doug Cutting and 
> the great Hadoop was";
>   String observed = highlightField(query, fieldName, theText);
>   System.out.println("Expected: \"" + expected + "\n" + "Observed: \"" + 
> observed);
>   assertEquals("Why is that second instance of the term \"Lucene\" 
> highlighted?", expected, observed);
> }
> Is this an issue that's arisen before?  I've been reading through the source 
> to QueryScorer, WeightedSpanTerm, WeightedSpanTermExtractor, Spans, and 
> NearSpansOrdered, but haven't found the solution yet.  Initially, I thought 
> that the extractWeightedSpanTerms method in WeightedSpanTermExtractor should 
> be called on each clause of a SpanNearQuery or SpanOrQuery, but that didn't 
> get me too far.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-4198) Allow codecs to index term impacts

[Adrien Grand (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adrien Grand updated LUCENE-4198:
-
Attachment: LUCENE-4198.patch

New patch. This time it has tests, does basic testing in CheckIndex and does 
not clone too much.

Results are very good on queries that score on a single term, almost too good, 
I'm currently thinking about how we could change the API to have something that 
is easier to propagate with boolean queries, even if it means term queries 
can't be as fast.

{noformat}
TaskQPS baseline  StdDev   QPS patch  StdDev
Pct diff
  AndHighLow 2050.37  (4.2%) 1745.54  (2.0%)  
-14.9% ( -20% -   -9%)
   OrHighLow  922.62  (3.7%)  844.54  (2.4%)   
-8.5% ( -14% -   -2%)
  AndHighMed  277.85  (1.8%)  258.11  (2.6%)   
-7.1% ( -11% -   -2%)
OrNotHighLow 1105.41  (3.6%) 1044.69  (2.0%)   
-5.5% ( -10% -0%)
 AndHighHigh  128.97  (1.1%)  121.89  (2.7%)   
-5.5% (  -9% -   -1%)
  Fuzzy2  166.62  (6.2%)  158.38  (6.3%)   
-4.9% ( -16% -8%)
   OrHighMed  177.56  (2.3%)  170.05  (1.9%)   
-4.2% (  -8% -0%)
  Fuzzy1  199.16  (4.4%)  193.05  (5.5%)   
-3.1% ( -12% -7%)
 MedSloppyPhrase   53.92  (2.2%)   52.40  (2.3%)   
-2.8% (  -7% -1%)
   LowPhrase  201.13  (1.7%)  195.87  (1.0%)   
-2.6% (  -5% -0%)
 LowSpanNear  363.85  (3.0%)  355.07  (2.5%)   
-2.4% (  -7% -3%)
  HighPhrase   62.68  (1.6%)   61.32  (1.2%)   
-2.2% (  -4% -0%)
   HighTermMonthSort  218.42  (9.8%)  214.35  (8.3%)   
-1.9% ( -18% -   18%)
 MedSpanNear   46.65  (1.4%)   45.89  (1.5%)   
-1.6% (  -4% -1%)
   MedPhrase  178.02  (1.5%)  175.24  (1.2%)   
-1.6% (  -4% -1%)
HighSpanNear   10.21  (3.4%)   10.11  (3.4%)   
-1.0% (  -7% -6%)
HighSloppyPhrase   32.32  (7.3%)   32.01  (7.1%)   
-1.0% ( -14% -   14%)
 LowSloppyPhrase   18.01  (2.7%)   17.85  (2.7%)   
-0.9% (  -6% -4%)
 Respell  320.99  (2.1%)  321.02  (2.4%)
0.0% (  -4% -4%)
  IntNRQ   29.29 (11.6%)   29.42 (12.5%)
0.4% ( -21% -   27%)
Wildcard  189.97  (4.6%)  191.87  (3.9%)
1.0% (  -7% -9%)
 Prefix3  166.43  (6.2%)  169.95  (5.4%)
2.1% (  -8% -   14%)
  OrHighHigh   48.00  (3.7%)   49.09  (3.9%)
2.3% (  -5% -   10%)
   HighTermDayOfYearSort  146.88  (7.4%)  150.76  (8.0%)
2.6% ( -11% -   19%)
 LowTerm  830.79  (2.6%) 2246.40  (9.9%)  
170.4% ( 153% -  187%)
OrNotHighMed  180.11  (1.5%) 1454.55 (15.7%)  
707.6% ( 680% -  735%)
 MedTerm  216.16  (1.7%) 3834.73 (37.0%) 
1674.0% (1608% - 1742%)
HighTerm  109.49  (2.0%) 1944.44 (45.3%) 
1675.9% (1597% - 1757%)
OrHighNotMed   57.55  (1.1%) 1292.66 (57.7%) 
2146.2% (2064% - 2229%)
OrHighNotLow   84.00  (1.1%) 1996.82 (75.4%) 
2277.2% (2176% - 2379%)
   OrNotHighHigh   58.22  (1.3%) 1479.53 (53.5%) 
2441.4% (2356% - 2528%)
   OrHighNotHigh   66.91  (1.2%) 2042.54 (55.1%) 
2952.6% (2862% - 3045%)
{noformat}

> Allow codecs to index term impacts
> --
>
> Key: LUCENE-4198
> URL: https://issues.apache.org/jira/browse/LUCENE-4198
> Project: Lucene - Core
>  Issue Type: Sub-task
>  Components: core/index
>Reporter: Robert Muir
> Attachments: LUCENE-4198.patch, LUCENE-4198.patch, LUCENE-4198.patch, 
> LUCENE-4198_flush.patch
>
>
> Subtask of LUCENE-4100.
> Thats an example of something similar to impact indexing (though, his 
> implementation currently stores a max for the entire term, the problem is the 
> same).
> We can imagine other similar algorithms too: I think the codec API should be 
> able to support these.
> Currently it really doesnt: Stefan worked around the problem by providing a 
> tool to 'rewrite' your index, he passes the IndexReader and Similarity to it. 
> But it would be better if we fixed the codec API.
> One problem is that the Postings writer needs to have access to the 
> Similarity. Another problem is that it needs access to the term and 
> collection statistics up front, rather than after the fact.
> This 

[jira] [Commented] (LUCENE-8119) Remove SimScorer.maxScore(maxFreq)

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16313193#comment-16313193
 ] 

Robert Muir commented on LUCENE-8119:
-

I don't understand the 0 stuff. This shouldn't happen right? At query-time we 
fixed sims so that they aren't ever asked to score non-existent fields/terms, 
to clear up all the divide-by-zero issues and make the api simpler. But this is 
an index-time API, the question is do we really invoke this when there are no 
terms? 

and why write a 0 value at all, if norms are supposed to be sparse we shouldnt 
call this method nor write anything, correct? we certainly won't be scoring any 
terms for the field on such documents, no value is needed. 

> Remove SimScorer.maxScore(maxFreq)
> --
>
> Key: LUCENE-8119
> URL: https://issues.apache.org/jira/browse/LUCENE-8119
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Adrien Grand
>Priority: Minor
> Attachments: LUCENE-8119.patch
>
>
> Now that SimScorer directly takes a frequency and a norm, this can be 
> replaced with {{SimScorer.freq(maxFreq, 1)}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11821) ConcurrentModificationException in SimSolrCloudTestCase.tearDown

[Shalin Shekhar Mangar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shalin Shekhar Mangar updated SOLR-11821:
-
Attachment: SOLR-11821.patch

Trivial fix to synchronize before iteration. I looked at the other uses of the 
system collection but they iterate only using indexed for loops so they seem 
safe.

> ConcurrentModificationException in SimSolrCloudTestCase.tearDown
> 
>
> Key: SOLR-11821
> URL: https://issues.apache.org/jira/browse/SOLR-11821
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: AutoScaling, Tests
>Reporter: Shalin Shekhar Mangar
> Fix For: master (8.0), 7.3
>
> Attachments: SOLR-11821.patch
>
>
> I noticed a few test failures on jenkins such as the following:
> {code}
> FAILED:  
> org.apache.solr.cloud.autoscaling.sim.TestComputePlanAction.testNodeLost
> Error Message:
> Stack Trace:
> java.util.ConcurrentModificationException
> at 
> __randomizedtesting.SeedInfo.seed([A41248828EFF34E3:1B07867C0D155165]:0)
> at 
> java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:937)
> at java.base/java.util.ArrayList$Itr.next(ArrayList.java:891)
> at 
> org.apache.solr.cloud.autoscaling.sim.SimSolrCloudTestCase.tearDown(SimSolrCloudTestCase.java:141)
> {code}
> The line in question iterates over a synchronized List but synchronized list 
> does not return a safe iterator.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-EA] Lucene-Solr-master-Linux (64bit/jdk-10-ea+37) - Build # 21218 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/21218/
Java: 64bit/jdk-10-ea+37 -XX:-UseCompressedOops -XX:+UseG1GC

3 tests failed.
FAILED:  org.apache.solr.cloud.BasicDistributedZk2Test.test

Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:45253

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting 
response from server at: http://127.0.0.1:45253
at 
__randomizedtesting.SeedInfo.seed([6A5F05FC518A82BF:E20B3A26FF76EF47]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:654)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:320)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:991)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[jira] [Updated] (LUCENE-8113) Allow terms dictionary lookups to be lazy when scores are not needed

[Alan Woodward (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-8113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Woodward updated LUCENE-8113:
--
Attachment: LUCENE-8113-rename.patch

Thanks for the review Mike.  Here's a patch with your suggested modifications, 
plus a CHANGES entry.  Will commit shortly.

> Allow terms dictionary lookups to be lazy when scores are not needed
> 
>
> Key: LUCENE-8113
> URL: https://issues.apache.org/jira/browse/LUCENE-8113
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Alan Woodward
> Attachments: LUCENE-8113-rename.patch, LUCENE-8113-rename.patch, 
> LUCENE-8113.patch, LUCENE-8113.patch, LUCENE-8113.patch
>
>
> LUCENE-7311 made it possible to avoid loading TermStates in cached 
> TermQueries.  It would be useful to extend this to other queries that use the 
> terms dictionary.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: BugFix release 7.2.1

[Adrien Grand]

+1

Looking at the changelog, 7.3 has 3 bug fixes for now: LUCENE-8077,
SOLR-11783 and SOLR-11555. The Lucene change doesn't seem worth
backporting, but maybe the Solr changes should?

Le ven. 5 janv. 2018 à 12:40, jim ferenczi  a
écrit :

> Hi,
> We discovered a bad bug in 7x that affects indices created in 6x with
> Lucene54DocValues format. The SortedNumericDocValues created with this
> format have a bug when advanceExact is used, the values retrieved for the
> docs when advanceExact returns true are invalid (the pointer to the values
> is not updated):
> https://issues.apache.org/jira/browse/LUCENE-8117
> This affects all indices created in 6x with sorted numeric doc values so I
> wanted to ask if anyone objects to a bugfix release for 7.2 (7.2.1). I also
> volunteer to be the release manager for this one if it is accepted.
>
> Jim
>

[jira] [Updated] (LUCENE-8119) Remove SimScorer.maxScore(maxFreq)

[Adrien Grand (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-8119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adrien Grand updated LUCENE-8119:
-
Attachment: LUCENE-8119.patch

Here is a patch. It documents some additional constraints on similarities, like 
the fact that norm 0 is reserved to empty fields and that scores must not 
increase when the unsigned norm increases. The latter is something I need for 
LUCENE-4198 as well.

> Remove SimScorer.maxScore(maxFreq)
> --
>
> Key: LUCENE-8119
> URL: https://issues.apache.org/jira/browse/LUCENE-8119
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Adrien Grand
>Priority: Minor
> Attachments: LUCENE-8119.patch
>
>
> Now that SimScorer directly takes a frequency and a norm, this can be 
> replaced with {{SimScorer.freq(maxFreq, 1)}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8113) Allow terms dictionary lookups to be lazy when scores are not needed

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16313083#comment-16313083
 ] 

Michael McCandless commented on LUCENE-8113:


I like the new name {{TermStates}}!

Can we change the exception message from:
{noformat}
throw new IllegalStateException("Cannot call docFreq() on lazily 
constructed TermStates");
{noformat}

to:
{noformat}
throw new IllegalStateException("Cannot call docFreq() when 
needsStats=false");
{noformat}

or so?

Also can you add { } around single-statement if bodies?  This is our code 
style, and leaving off the { } adds risk to future refactoring.  Thanks ;)


> Allow terms dictionary lookups to be lazy when scores are not needed
> 
>
> Key: LUCENE-8113
> URL: https://issues.apache.org/jira/browse/LUCENE-8113
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Alan Woodward
> Attachments: LUCENE-8113-rename.patch, LUCENE-8113.patch, 
> LUCENE-8113.patch, LUCENE-8113.patch
>
>
> LUCENE-7311 made it possible to avoid loading TermStates in cached 
> TermQueries.  It would be useful to extend this to other queries that use the 
> terms dictionary.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-8119) Remove SimScorer.maxScore(maxFreq)

[Adrien Grand (JIRA)]

Adrien Grand created LUCENE-8119:


 Summary: Remove SimScorer.maxScore(maxFreq)
 Key: LUCENE-8119
 URL: https://issues.apache.org/jira/browse/LUCENE-8119
 Project: Lucene - Core
  Issue Type: Task
Reporter: Adrien Grand
Priority: Minor


Now that SimScorer directly takes a frequency and a norm, this can be replaced 
with {{SimScorer.freq(maxFreq, 1)}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-9675) Sorting on field in JSON Facet API which is not part of JSON Facet.

[Amrit Sarkar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amrit Sarkar resolved SOLR-9675.

Resolution: Not A Problem

> Sorting on field in JSON Facet API which is not part of JSON Facet.
> ---
>
> Key: SOLR-9675
> URL: https://issues.apache.org/jira/browse/SOLR-9675
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Amrit Sarkar
>Priority: Minor
>
> Here's a sample example:
> There is a requirement to facet on a particular field but sort on another 
> field which is not part of json facet.
> For example, consider schema with fields : sl1, sl2, product_bkgs, gc_2
> Solr query & facet : q=sl1 : ("abc") AND sl2 : ("xyz")=sl1 desc=0
> & json.facet={
> "group_column_level" :
> {
> "type" : "terms",
> "field" : "gc_2",
> "offset" : 0,
> "limit" :25,
> "sort" : { "product_bkgs" : "desc"},
> "facet" :
> {
> "product_bkgs" :"sum(product_bkgs)"
> }
> }
> }
> Sort on product_bkgs is possible but not on sl1 in the facet.
> Let me know if anything can be done to achieve the same.
> Thanks in advance.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-EA] Lucene-Solr-7.x-Linux (64bit/jdk-10-ea+37) - Build # 1123 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/1123/
Java: 64bit/jdk-10-ea+37 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC

3 tests failed.
FAILED:  org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test

Error Message:
Expected 2 of 3 replicas to be active but only found 1; 
[core_node4:{"core":"c8n_1x3_lf_shard1_replica_n1","base_url":"http://127.0.0.1:46053/x","node_name":"127.0.0.1:46053_x","state":"active","type":"NRT","leader":"true"}];
 clusterState: 
DocCollection(c8n_1x3_lf//collections/c8n_1x3_lf/state.json/16)={   
"pullReplicas":"0",   "replicationFactor":"1",   "shards":{"shard1":{   
"range":"8000-7fff",   "state":"active",   "replicas":{ 
"core_node4":{   "core":"c8n_1x3_lf_shard1_replica_n1",   
"base_url":"http://127.0.0.1:46053/x;,   
"node_name":"127.0.0.1:46053_x",   "state":"active",   
"type":"NRT",   "leader":"true"}, "core_node5":{   
"core":"c8n_1x3_lf_shard1_replica_n3",   
"base_url":"http://127.0.0.1:36797/x;,   
"node_name":"127.0.0.1:36797_x",   "state":"down",   
"type":"NRT"}, "core_node6":{   "state":"down",   
"base_url":"http://127.0.0.1:42425/x;,   
"core":"c8n_1x3_lf_shard1_replica_n2",   
"node_name":"127.0.0.1:42425_x",   "type":"NRT",   
"router":{"name":"compositeId"},   "maxShardsPerNode":"1",   
"autoAddReplicas":"false",   "nrtReplicas":"3",   "tlogReplicas":"0"}

Stack Trace:
java.lang.AssertionError: Expected 2 of 3 replicas to be active but only found 
1; 
[core_node4:{"core":"c8n_1x3_lf_shard1_replica_n1","base_url":"http://127.0.0.1:46053/x","node_name":"127.0.0.1:46053_x","state":"active","type":"NRT","leader":"true"}];
 clusterState: DocCollection(c8n_1x3_lf//collections/c8n_1x3_lf/state.json/16)={
  "pullReplicas":"0",
  "replicationFactor":"1",
  "shards":{"shard1":{
  "range":"8000-7fff",
  "state":"active",
  "replicas":{
"core_node4":{
  "core":"c8n_1x3_lf_shard1_replica_n1",
  "base_url":"http://127.0.0.1:46053/x;,
  "node_name":"127.0.0.1:46053_x",
  "state":"active",
  "type":"NRT",
  "leader":"true"},
"core_node5":{
  "core":"c8n_1x3_lf_shard1_replica_n3",
  "base_url":"http://127.0.0.1:36797/x;,
  "node_name":"127.0.0.1:36797_x",
  "state":"down",
  "type":"NRT"},
"core_node6":{
  "state":"down",
  "base_url":"http://127.0.0.1:42425/x;,
  "core":"c8n_1x3_lf_shard1_replica_n2",
  "node_name":"127.0.0.1:42425_x",
  "type":"NRT",
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"1",
  "autoAddReplicas":"false",
  "nrtReplicas":"3",
  "tlogReplicas":"0"}
at 
__randomizedtesting.SeedInfo.seed([E4866648AB0CCE26:6CD2599205F0A3DE]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.testRf3WithLeaderFailover(LeaderFailoverAfterPartitionTest.java:169)
at 
org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test(LeaderFailoverAfterPartitionTest.java:56)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 

[jira] [Resolved] (SOLR-11652) Cdcr TLogs doesn't get purged for Source collection Leader when Buffer is disabled from CDCR API

[Amrit Sarkar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amrit Sarkar resolved SOLR-11652.
-
Resolution: Later

> Cdcr TLogs doesn't get purged for Source collection Leader when Buffer is 
> disabled from CDCR API
> 
>
> Key: SOLR-11652
> URL: https://issues.apache.org/jira/browse/SOLR-11652
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Amrit Sarkar
> Attachments: SOLR-11652.patch
>
>
> Cdcr transactions logs doesn't get purged on leader EVER when Buffer DISABLED 
> from CDCR API.
> Steps to reproduce:
> 1. Setup source and target collection cluster and START CDCR, BUFFER ENABLED.
> 2. Index bunch of documents into source; make sure we have generated tlogs in 
> decent numbers (>20)
> 3. Disable BUFFER via API on source and keep on indexing
> 4. Tlogs starts to get purges on follower nodes of Source, but Leader keeps 
> on accumulating ever.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11671) CdcrUpdateLog should be enabled smartly for Cdcr configured collection

[Amrit Sarkar (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16313013#comment-16313013
 ] 

Amrit Sarkar commented on SOLR-11671:
-

This will be part of SOLR-8389.

> CdcrUpdateLog should be enabled smartly for Cdcr configured collection
> --
>
> Key: SOLR-11671
> URL: https://issues.apache.org/jira/browse/SOLR-11671
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: CDCR
>Affects Versions: 7.2
>Reporter: Amrit Sarkar
> Attachments: SOLR-11671.patch
>
>
> {{CdcrUpdateLog}} should be configured smartly by itself when collection 
> config has *CDCR Request Handler* specified.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-8113) Allow terms dictionary lookups to be lazy when scores are not needed

[Alan Woodward (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-8113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Woodward updated LUCENE-8113:
--
Attachment: LUCENE-8113-rename.patch

OK, here's the patch again, this time including the rename (which makes it 
pretty big)

> Allow terms dictionary lookups to be lazy when scores are not needed
> 
>
> Key: LUCENE-8113
> URL: https://issues.apache.org/jira/browse/LUCENE-8113
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Alan Woodward
> Attachments: LUCENE-8113-rename.patch, LUCENE-8113.patch, 
> LUCENE-8113.patch, LUCENE-8113.patch
>
>
> LUCENE-7311 made it possible to avoid loading TermStates in cached 
> TermQueries.  It would be useful to extend this to other queries that use the 
> terms dictionary.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (LUCENE-8118) ArrayIndexOutOfBoundsException in TermsHashPerField.writeByte during indexing

[Diego Ceccarelli (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16312971#comment-16312971
 ] 

Diego Ceccarelli edited comment on LUCENE-8118 at 1/5/18 11:53 AM:
---

Looking at your code it seems that there is only one commit at the end, and 
your collection is big. What if you try to commit every, let's say, 50k docs?  


was (Author: diegoceccarelli):
Looking at your code it seems that there is only one commit at the end, and 
your collection is big. Could you please try to commit every, let's say, 50k 
docs?  

> ArrayIndexOutOfBoundsException in TermsHashPerField.writeByte during indexing
> -
>
> Key: LUCENE-8118
> URL: https://issues.apache.org/jira/browse/LUCENE-8118
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index
>Affects Versions: 7.2
> Environment: Debian/Stretch
> java version "1.8.0_144"  
>   
>Java(TM) SE Runtime 
> Environment (build 1.8.0_144-b01) 
>   
>Java HotSpot(TM) 64-Bit Server VM (build 
> 25.144-b01, mixed mode)
>Reporter: Laura Dietz
>
> Indexing a large collection of about 20 million paragraph-sized documents 
> results in an ArrayIndexOutOfBoundsException in 
> org.apache.lucene.index.TermsHashPerField.writeByte  (full stack trace 
> below). 
> The bug is possibly related to issues described in 
> [here|http://lucene.472066.n3.nabble.com/ArrayIndexOutOfBoundsException-65536-td3661945.html]
>   and [SOLR-10936|https://issues.apache.org/jira/browse/SOLR-10936] -- but I 
> am not using SOLR, I am directly using Lucene Core.
> The issue can be reproduced using code from  [GitHub 
> trec-car-tools-example|https://github.com/TREMA-UNH/trec-car-tools/tree/lucene-bug/trec-car-tools-example]
>  
> - compile with `mvn compile assembly:single`
> - run with `java -cp 
> ./target/treccar-tools-example-0.1-jar-with-dependencies.jar 
> edu.unh.cs.TrecCarBuildLuceneIndex paragraphs paragraphCorpus.cbor indexDir`
> Where paragraphCorpus.cbor is contained in this 
> [archive|http://trec-car.cs.unh.edu/datareleases/v2.0-snapshot/archive-paragraphCorpus.tar.xz]
> Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -65536   
> at 
> org.apache.lucene.index.TermsHashPerField.writeByte(TermsHashPerField.java:198)
>   
>at 
> org.apache.lucene.index.TermsHashPerField.writeVInt(TermsHashPerField.java:224)
>   
>at 
> org.apache.lucene.index.FreqProxTermsWriterPerField.addTerm(FreqProxTermsWriterPerField.java:159)
>   
>  at 
> org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:185) 
>   
> at 
> org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:786)
>   
>at 
> org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:430)
>   
>   at 
> org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:392)
>   
>at 
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:281)
>   
>at 
> org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:451)
>   
>  at 
> org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1532)
>   
> at 
> 

BugFix release 7.2.1

[jim ferenczi]

Hi,
We discovered a bad bug in 7x that affects indices created in 6x with
Lucene54DocValues format. The SortedNumericDocValues created with this
format have a bug when advanceExact is used, the values retrieved for the
docs when advanceExact returns true are invalid (the pointer to the values
is not updated):
https://issues.apache.org/jira/browse/LUCENE-8117
This affects all indices created in 6x with sorted numeric doc values so I
wanted to ask if anyone objects to a bugfix release for 7.2 (7.2.1). I also
volunteer to be the release manager for this one if it is accepted.

Jim

[jira] [Resolved] (LUCENE-8117) advanceExact does not work on sorted numeric dvs with Lucene54DocValuesProducer

[Jim Ferenczi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-8117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jim Ferenczi resolved LUCENE-8117.
--
   Resolution: Fixed
Fix Version/s: 7.3
   7.2.1

> advanceExact does not work on sorted numeric dvs with 
> Lucene54DocValuesProducer 
> 
>
> Key: LUCENE-8117
> URL: https://issues.apache.org/jira/browse/LUCENE-8117
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 7.2
>Reporter: Jim Ferenczi
> Fix For: 7.2.1, 7.3
>
> Attachments: LUCENE-8117.patch
>
>
> DocValues are iterators now so old doc values (produced with 
> Lucene54DocValues) also implements advance and advanceExact. Though sorted 
> numerics produced by Lucene54DocValues are not working as expected when 
> advanceExact is used. 
> In such case, the docValueCount is as expected but the values returned by the 
> iterator for the document are invalid. This is due to a bug in the 
> implementation of advanceExact in the producer that does not set the offset 
> of the current doc when the function is used.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8117) advanceExact does not work on sorted numeric dvs with Lucene54DocValuesProducer

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16312993#comment-16312993
 ] 

ASF subversion and git services commented on LUCENE-8117:
-

Commit 420461337070b3316634e5ac16a5b65cc99e8f87 in lucene-solr's branch 
refs/heads/branch_7_2 from [~jimczi]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=4204613 ]

LUCENE-8117: Fix advanceExact on SortedNumericDocValues produced by 
Lucene54DocValues.


> advanceExact does not work on sorted numeric dvs with 
> Lucene54DocValuesProducer 
> 
>
> Key: LUCENE-8117
> URL: https://issues.apache.org/jira/browse/LUCENE-8117
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 7.2
>Reporter: Jim Ferenczi
> Fix For: 7.3, 7.2.1
>
> Attachments: LUCENE-8117.patch
>
>
> DocValues are iterators now so old doc values (produced with 
> Lucene54DocValues) also implements advance and advanceExact. Though sorted 
> numerics produced by Lucene54DocValues are not working as expected when 
> advanceExact is used. 
> In such case, the docValueCount is as expected but the values returned by the 
> iterator for the document are invalid. This is due to a bug in the 
> implementation of advanceExact in the producer that does not set the offset 
> of the current doc when the function is used.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8117) advanceExact does not work on sorted numeric dvs with Lucene54DocValuesProducer

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16312991#comment-16312991
 ] 

ASF subversion and git services commented on LUCENE-8117:
-

Commit 0b417545774e7b0a93aaab16ece0d50836b729d9 in lucene-solr's branch 
refs/heads/branch_7x from [~jimczi]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=0b41754 ]

LUCENE-8117: Fix advanceExact on SortedNumericDocValues produced by 
Lucene54DocValues.


> advanceExact does not work on sorted numeric dvs with 
> Lucene54DocValuesProducer 
> 
>
> Key: LUCENE-8117
> URL: https://issues.apache.org/jira/browse/LUCENE-8117
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 7.2
>Reporter: Jim Ferenczi
> Attachments: LUCENE-8117.patch
>
>
> DocValues are iterators now so old doc values (produced with 
> Lucene54DocValues) also implements advance and advanceExact. Though sorted 
> numerics produced by Lucene54DocValues are not working as expected when 
> advanceExact is used. 
> In such case, the docValueCount is as expected but the values returned by the 
> iterator for the document are invalid. This is due to a bug in the 
> implementation of advanceExact in the producer that does not set the offset 
> of the current doc when the function is used.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-7.x-MacOSX (64bit/jdk-9) - Build # 381 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-MacOSX/381/
Java: 64bit/jdk-9 -XX:+UseCompressedOops -XX:+UseParallelGC

72 tests failed.
FAILED:  org.apache.solr.TestCursorMarkWithoutUniqueKey.test

Error Message:
invalid API spec: apispec/cluster.security.authorization.json

Stack Trace:
java.lang.RuntimeException: invalid API spec: 
apispec/cluster.security.authorization.json
at 
__randomizedtesting.SeedInfo.seed([84ACFE814398196D:CF8C15BED647495]:0)
at 
org.apache.solr.common.util.ValidatingJsonMap.parse(ValidatingJsonMap.java:318)
at org.apache.solr.common.util.Utils.lambda$getSpec$0(Utils.java:427)
at org.apache.solr.api.Api.getSpec(Api.java:65)
at org.apache.solr.api.ApiBag.register(ApiBag.java:73)
at org.apache.solr.core.PluginBag.put(PluginBag.java:217)
at org.apache.solr.core.PluginBag.put(PluginBag.java:188)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:541)
at org.apache.solr.util.TestHarness.(TestHarness.java:178)
at org.apache.solr.util.TestHarness.(TestHarness.java:141)
at org.apache.solr.util.TestHarness.(TestHarness.java:147)
at org.apache.solr.util.TestHarness.(TestHarness.java:110)
at org.apache.solr.SolrTestCaseJ4.createCore(SolrTestCaseJ4.java:705)
at org.apache.solr.SolrTestCaseJ4.initCore(SolrTestCaseJ4.java:695)
at org.apache.solr.SolrTestCaseJ4.initCore(SolrTestCaseJ4.java:565)
at org.apache.solr.SolrTestCaseJ4.initCore(SolrTestCaseJ4.java:554)
at 
org.apache.solr.TestCursorMarkWithoutUniqueKey.beforeSetupCore(TestCursorMarkWithoutUniqueKey.java:38)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:968)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[jira] [Commented] (LUCENE-8118) ArrayIndexOutOfBoundsException in TermsHashPerField.writeByte during indexing

[Diego Ceccarelli (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16312971#comment-16312971
 ] 

Diego Ceccarelli commented on LUCENE-8118:
--

Looking at your code it seems that there is only one commit at the end, and 
your collection is big. Could you please try to commit every, let's say, 50k 
docs?  

> ArrayIndexOutOfBoundsException in TermsHashPerField.writeByte during indexing
> -
>
> Key: LUCENE-8118
> URL: https://issues.apache.org/jira/browse/LUCENE-8118
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index
>Affects Versions: 7.2
> Environment: Debian/Stretch
> java version "1.8.0_144"  
>   
>Java(TM) SE Runtime 
> Environment (build 1.8.0_144-b01) 
>   
>Java HotSpot(TM) 64-Bit Server VM (build 
> 25.144-b01, mixed mode)
>Reporter: Laura Dietz
>
> Indexing a large collection of about 20 million paragraph-sized documents 
> results in an ArrayIndexOutOfBoundsException in 
> org.apache.lucene.index.TermsHashPerField.writeByte  (full stack trace 
> below). 
> The bug is possibly related to issues described in 
> [here|http://lucene.472066.n3.nabble.com/ArrayIndexOutOfBoundsException-65536-td3661945.html]
>   and [SOLR-10936|https://issues.apache.org/jira/browse/SOLR-10936] -- but I 
> am not using SOLR, I am directly using Lucene Core.
> The issue can be reproduced using code from  [GitHub 
> trec-car-tools-example|https://github.com/TREMA-UNH/trec-car-tools/tree/lucene-bug/trec-car-tools-example]
>  
> - compile with `mvn compile assembly:single`
> - run with `java -cp 
> ./target/treccar-tools-example-0.1-jar-with-dependencies.jar 
> edu.unh.cs.TrecCarBuildLuceneIndex paragraphs paragraphCorpus.cbor indexDir`
> Where paragraphCorpus.cbor is contained in this 
> [archive|http://trec-car.cs.unh.edu/datareleases/v2.0-snapshot/archive-paragraphCorpus.tar.xz]
> Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -65536   
> at 
> org.apache.lucene.index.TermsHashPerField.writeByte(TermsHashPerField.java:198)
>   
>at 
> org.apache.lucene.index.TermsHashPerField.writeVInt(TermsHashPerField.java:224)
>   
>at 
> org.apache.lucene.index.FreqProxTermsWriterPerField.addTerm(FreqProxTermsWriterPerField.java:159)
>   
>  at 
> org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:185) 
>   
> at 
> org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:786)
>   
>at 
> org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:430)
>   
>   at 
> org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:392)
>   
>at 
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:281)
>   
>at 
> org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:451)
>   
>  at 
> org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1532)
>   
> at 
> org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1508)
> at 
> edu.unh.cs.TrecCarBuildLuceneIndex.main(TrecCarBuildLuceneIndex.java:55)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (LUCENE-8113) Allow terms dictionary lookups to be lazy when scores are not needed

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16312968#comment-16312968
 ] 

Robert Muir commented on LUCENE-8113:
-

Wait, I don't think thats right. 

There's no bug here, so there is no need to urgently backport any fix. No need 
to sacrifice code quality in the name of backporting.

I dont think this change should be backported at all. Lets do it correctly for 
master-only instead.

> Allow terms dictionary lookups to be lazy when scores are not needed
> 
>
> Key: LUCENE-8113
> URL: https://issues.apache.org/jira/browse/LUCENE-8113
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Alan Woodward
> Attachments: LUCENE-8113.patch, LUCENE-8113.patch, LUCENE-8113.patch
>
>
> LUCENE-7311 made it possible to avoid loading TermStates in cached 
> TermQueries.  It would be useful to extend this to other queries that use the 
> terms dictionary.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org