date:20161209

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/18492/
Java: 32bit/jdk-9-ea+147 -client -XX:+UseG1GC

1 tests failed.
FAILED:  org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.test

Error Message:
document count mismatch.  control=1673 sum(shards)=1672 cloudClient=1672

Stack Trace:
java.lang.AssertionError: document count mismatch.  control=1673 
sum(shards)=1672 cloudClient=1672
at 
__randomizedtesting.SeedInfo.seed([6D67B60F8D1C1116:E53389D523E07CEE]:0)
at org.junit.Assert.fail(Assert.java:93)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1323)
at 
org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.test(ChaosMonkeyNothingIsSafeTest.java:231)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:538)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at

[jira] [Commented] (SOLR-9841) Disable expiration processor commits

2016-12-09 Thread Alexandre Rafalovitch (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-9841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15737415#comment-15737415
 ] 

Alexandre Rafalovitch commented on SOLR-9841:
-

bq. They added a dropdown arrow on the Create button, and if you click that 
arrow, the only option is to create a service desk request. If you click on the 
left side of the button instead, it works just like it always has. I filed an 
issue with Atlassian about it, but it got closed and I don't think they'll be 
changing it.

I think Infra enabled ServiceDesk component for JIRA on the last upgrade which 
is why we suddenly got it. And - mea culpa - I actually enabled Service Desk 
for Solr to see whether the reporting abilities it had would be useful for 
"dead issues" tracking. I did not realize it would then show big red numbers 
everywhere as well. I've disabled that for Solr just now, but this still does 
not fix the 'Add Service Request' issue, which IIRC was already there before. 
So, if this needs to be escalated, it should probably go into Infra hands (and 
separate issue). 

> Disable expiration processor commits
> 
>
> Key: SOLR-9841
> URL: https://issues.apache.org/jira/browse/SOLR-9841
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: Brent Pearson
>
> http://lucene.472066.n3.nabble.com/Adding-DocExpirationUpdateProcessorFactory-causes-quot-Overlapping-onDeckSearchers-quot-warnings-td4309155.html
> Need a way to have document expiration update processor *not* trigger 
> additional commits.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-6.x-MacOSX (64bit/jdk1.8.0) - Build # 566 - Unstable!

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-6.x-MacOSX/566/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.cloud.DocValuesNotIndexedTest.testGroupingDVOnly

Error Message:
Unexpected number of elements in the group for intGSF: 9

Stack Trace:
java.lang.AssertionError: Unexpected number of elements in the group for 
intGSF: 9
at 
__randomizedtesting.SeedInfo.seed([BECB0ED5D3C30C3:9057DEB51064029D]:0)
at org.junit.Assert.fail(Assert.java:93)
at 
org.apache.solr.cloud.DocValuesNotIndexedTest.testGroupingDVOnly(DocValuesNotIndexedTest.java:371)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 12147 lines...]
   [junit4] Suite: org.apache.solr.cloud.DocValuesNotIndexedTest
   [junit4]   2> Creating dataDir:

[jira] [Commented] (SOLR-5882) Introduce score local parameter for {!parent} query parser

2016-12-09 Thread Mikhail Khludnev (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15737375#comment-15737375
 ] 

Mikhail Khludnev commented on SOLR-5882:


It should, I believe. Could you provide a specific example including parsed 
query from debugQuery=true?

> Introduce score local parameter for {!parent} query parser
> --
>
> Key: SOLR-5882
> URL: https://issues.apache.org/jira/browse/SOLR-5882
> Project: Solr
>  Issue Type: New Feature
>Affects Versions: 4.8
>Reporter: Andrey Kudryavtsev
>Assignee: Mikhail Khludnev
> Fix For: 5.3
>
> Attachments: SOLR-5882.patch, SOLR-5882.patch
>
>
> I propose to have ability to configure scoring mode by *optional* local 
> parameter
> Syntax for parent queries is:
> {code:borderStyle=solid}
> {!parent which=type:parent score=none|avg|max|total|min}...
> Capital case for score values is also accepted
> {code} 
> Child query enables scoring by default.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9841) Disable expiration processor commits

2016-12-09 Thread Shawn Heisey (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-9841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15737326#comment-15737326
 ] 

Shawn Heisey commented on SOLR-9841:


bq. Actually .. i take that back ... jira does appear to have been 
changed/upgraded in some way so the only "Create" option is "Create Service 
Desk Request"

They added a dropdown arrow on the Create button, and if you click that arrow, 
the only option is to create a service desk request.  If you click on the left 
side of the button instead, it works just like it always has.  I filed an issue 
with Atlassian about it, but it got closed and I don't think they'll be 
changing it.


> Disable expiration processor commits
> 
>
> Key: SOLR-9841
> URL: https://issues.apache.org/jira/browse/SOLR-9841
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: Brent Pearson
>
> http://lucene.472066.n3.nabble.com/Adding-DocExpirationUpdateProcessorFactory-causes-quot-Overlapping-onDeckSearchers-quot-warnings-td4309155.html
> Need a way to have document expiration update processor *not* trigger 
> additional commits.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-7575) UnifiedHighlighter: add requireFieldMatch=false support

2016-12-09 Thread David Smiley (JIRA)


 [ 
https://issues.apache.org/jira/browse/LUCENE-7575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Smiley resolved LUCENE-7575.
--
Resolution: Fixed

> UnifiedHighlighter: add requireFieldMatch=false support
> ---
>
> Key: LUCENE-7575
> URL: https://issues.apache.org/jira/browse/LUCENE-7575
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/highlighter
>Reporter: David Smiley
>Assignee: David Smiley
> Fix For: 6.4
>
> Attachments: LUCENE-7575.patch, LUCENE-7575.patch, LUCENE-7575.patch
>
>
> The UnifiedHighlighter (like the PostingsHighlighter) only supports 
> highlighting queries for the same fields that are being highlighted.  The 
> original Highlighter and FVH support loosening this, AKA 
> requireFieldMatch=false.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7575) UnifiedHighlighter: add requireFieldMatch=false support

2016-12-09 Thread David Smiley (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-7575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15737220#comment-15737220
 ] 

David Smiley commented on LUCENE-7575:
--

Hoss notified me of my screw-up on the dev list where I replied to him and told 
him I fixed it (on the 6th):
http://mail-archives.apache.org/mod_mbox/lucene-dev/201612.mbox/%3ccabewpvf-qjiuy7a4ft-eumg4j2ptvufoldjukkdvouxqas+...@mail.gmail.com%3e
Here's the commit to CHANGES.txt  
https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;a=blobdiff;f=lucene/CHANGES.txt;h=21a80e00619d6723fc6d163b5068eae6f25b097f;hp=853f17125b938559c3fab94a8d1aca91a620e29e;hb=8c943b6084ffec3e22b16f4997df1f9fc55f20f5;hpb=135604f6327032d0258227aaa524369203d40822
Perhaps I should have referenced the issue but the change/correction itself was 
CHANGES.txt housekeeping that wasn't actually about any particular issue.

> UnifiedHighlighter: add requireFieldMatch=false support
> ---
>
> Key: LUCENE-7575
> URL: https://issues.apache.org/jira/browse/LUCENE-7575
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/highlighter
>Reporter: David Smiley
>Assignee: David Smiley
> Fix For: 6.4
>
> Attachments: LUCENE-7575.patch, LUCENE-7575.patch, LUCENE-7575.patch
>
>
> The UnifiedHighlighter (like the PostingsHighlighter) only supports 
> highlighting queries for the same fields that are being highlighted.  The 
> original Highlighter and FVH support loosening this, AKA 
> requireFieldMatch=false.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-6.x-Windows (32bit/jdk1.8.0_112) - Build # 612 - Unstable!

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Windows/612/
Java: 32bit/jdk1.8.0_112 -server -XX:+UseParallelGC

1 tests failed.
FAILED:  
org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigAliasReplication

Error Message:
expected:<1> but was:<0>

Stack Trace:
java.lang.AssertionError: expected:<1> but was:<0>
at 
__randomizedtesting.SeedInfo.seed([EDCE5BA4B1049DEF:1ABDB5FC77EC3209]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at 
org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigAliasReplication(TestReplicationHandler.java:1331)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11019 lines...]
   [junit4] Suite:

[JENKINS] Lucene-Solr-6.x-Linux (32bit/jdk1.8.0_112) - Build # 2383 - Unstable!

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/2383/
Java: 32bit/jdk1.8.0_112 -client -XX:+UseSerialGC

1 tests failed.
FAILED:  
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI

Error Message:
expected:<3> but was:<1>

Stack Trace:
java.lang.AssertionError: expected:<3> but was:<1>
at 
__randomizedtesting.SeedInfo.seed([81ACC65C8A3860B0:C9D9B2E88C0B4F25]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:516)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11480 lines...]
   [junit4] Suite:

[jira] [Commented] (SOLR-5882) Introduce score local parameter for {!parent} query parser

2016-12-09 Thread Vladimir Strugatsky (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15737035#comment-15737035
 ] 

Vladimir Strugatsky commented on SOLR-5882:
---

It appears that although "score" is available in the parent document, "boost" 
function has no effect when using the {!parent} query parser.

> Introduce score local parameter for {!parent} query parser
> --
>
> Key: SOLR-5882
> URL: https://issues.apache.org/jira/browse/SOLR-5882
> Project: Solr
>  Issue Type: New Feature
>Affects Versions: 4.8
>Reporter: Andrey Kudryavtsev
>Assignee: Mikhail Khludnev
> Fix For: 5.3
>
> Attachments: SOLR-5882.patch, SOLR-5882.patch
>
>
> I propose to have ability to configure scoring mode by *optional* local 
> parameter
> Syntax for parent queries is:
> {code:borderStyle=solid}
> {!parent which=type:parent score=none|avg|max|total|min}...
> Capital case for score values is also accepted
> {code} 
> Child query enables scoring by default.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9841) Disable expiration processor commits


[ 
https://issues.apache.org/jira/browse/SOLR-9841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736910#comment-15736910
 ] 

Hoss Man commented on SOLR-9841:


bq. Well, that's not the original issue, is it? The problem is that we're 
opening multiple searchers in rapid succession apparently due to autocommit 
expiry coupled with this commit. ...

I've seen no indication/confirmation of that -- the original mail thread that 
spawned this issue noted only that multiple on deck searchers as the result of 
using the doc expiration processor with a autoDeletePeriodSeconds=30 -- if 
warming takes more then 30 seconds *AND* docs are expiring at a frequency of at 
least 30 seconds, then that would explaining the multiple on deck searchers, 
even if there were no other adds/commits.

bq. ... maybe I'm jumping the gun here and the real question is whether the 
commit from the code resets the autocommit expiry?

All commits reset the autocommit timmer, regardless of when/why it was started.

bq. We can quibble about this one I suppose. In many cases it's probably good 
enough to let the next autocommit take care of opening a new searcher and 
rendering the docs un-findable rather than forcing the commit from the code. 
This assumes that the delete-by-query starts an autocommit timer.

I suppose the processor could try to detect if autocommit is enabled, and only 
use commitWithin on the delete instead of triggering the softCommit directly? 
... but i really think adding yet another config option that people might get 
wrong (ie: if they aren't using autoCommit) is a bad idea.

the bottom line is: if the processor is triggering on deck warming warnings 
when configured to run every 30 seconds, then even if it didn't trigger commits 
directly there would still be on deck warming warnings if the autoCommit is <= 
30 seconds.


> Disable expiration processor commits
> 
>
> Key: SOLR-9841
> URL: https://issues.apache.org/jira/browse/SOLR-9841
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: Brent Pearson
>
> http://lucene.472066.n3.nabble.com/Adding-DocExpirationUpdateProcessorFactory-causes-quot-Overlapping-onDeckSearchers-quot-warnings-td4309155.html
> Need a way to have document expiration update processor *not* trigger 
> additional commits.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9841) Disable expiration processor commits

2016-12-09 Thread Erick Erickson (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-9841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736889#comment-15736889
 ] 

Erick Erickson commented on SOLR-9841:
--

[~hoss...@fucit.org]

bq: The fact that there is no option prevent any commit at all was a conscious 
choice:

We can quibble about this one I suppose. In many cases it's probably good 
enough to let the next autocommit take care of opening a new searcher and 
rendering the docs un-findable rather than forcing the commit from the code. 
This assumes that the delete-by-query starts an autocommit timer.

If the decision is that we shouldn't/won't make this configurable then we 
should at least change the TODOs

// TODO: should this be more configurable? 
// TODO: in particular: should hard commit be optional?
since they make it seem like something that's still an open question. And yes I 
see the distinction between soft and hard commit implicit there.

bq: there is no value add in a new option to prevent the commit from being 
fired – it won't do anything unless there is something to commit.

Well, that's not the original issue, is it? The problem is that we're opening 
multiple searchers in rapid succession apparently due to autocommit expiry 
coupled with this commit. H, maybe I'm jumping the gun here and the real 
question is whether the commit from the code resets the autocommit expiry? I 
admit I haven't tracked that bit down. 

I suppose another sequence could be:
- autocommit expires and starts warming a new searcher
- TTL kicks in, deletes some docs and does the commit while the searcher from 
<1> is still warming producing the warning.

All that said, this doesn't seem like a major problem more a tidying-up.

Hmmm, when I hover over the "Create" button the tooltip is "create a new 
issue/bug/feature request", but the drop down is, indeed, Create Service Desk 
Request". Sigh.



> Disable expiration processor commits
> 
>
> Key: SOLR-9841
> URL: https://issues.apache.org/jira/browse/SOLR-9841
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: Brent Pearson
>
> http://lucene.472066.n3.nabble.com/Adding-DocExpirationUpdateProcessorFactory-causes-quot-Overlapping-onDeckSearchers-quot-warnings-td4309155.html
> Need a way to have document expiration update processor *not* trigger 
> additional commits.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9841) Disable expiration processor commits


[ 
https://issues.apache.org/jira/browse/SOLR-9841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736818#comment-15736818
 ] 

Hoss Man commented on SOLR-9841:


bq. yes you did in fact create a jira issue – aparently jira was upgraded 
recently and the term "Issue" doesn't seem to exist anymore, they appear to 
have been renamed into "Service Desk Request" ... i don't know why.

Actually .. i take that back ... jira does appear to have been changed/upgraded 
in some way so the only "Create" option is "Create Service Desk Request", and 
the result _seems_ like the same as creating an issue, but the look of this 
"issue" is totally diff from any other existing issue ... so i'm not sure what 
exactly is going on.

> Disable expiration processor commits
> 
>
> Key: SOLR-9841
> URL: https://issues.apache.org/jira/browse/SOLR-9841
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: Brent Pearson
>
> http://lucene.472066.n3.nabble.com/Adding-DocExpirationUpdateProcessorFactory-causes-quot-Overlapping-onDeckSearchers-quot-warnings-td4309155.html
> Need a way to have document expiration update processor *not* trigger 
> additional commits.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7589) Prevent outliers from raising the number of bits of everyone with numeric doc values


[ 
https://issues.apache.org/jira/browse/LUCENE-7589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736776#comment-15736776
 ] 

Michael McCandless commented on LUCENE-7589:


The patch looks great, just this minor typo:

{{ values for te next block.}} --> {{ values for the next block.}}

This seems to give ~3.7% reduction in the doc values disk used for sparse taxis!

> Prevent outliers from raising the number of bits of everyone with numeric doc 
> values
> 
>
> Key: LUCENE-7589
> URL: https://issues.apache.org/jira/browse/LUCENE-7589
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Adrien Grand
>Assignee: Adrien Grand
>Priority: Minor
> Attachments: LUCENE-7589.patch
>
>
> Today we encode entire segments with a single number of bits per value. It 
> was done this way because it was faster, but it also means a single outlier 
> can significantly increase the space requirements. I think we should have 
> protection against that.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9841) Disable expiration processor commits


[ 
https://issues.apache.org/jira/browse/SOLR-9841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736765#comment-15736765
 ] 

Hoss Man commented on SOLR-9841:


>From my mailing list reply...

{quote}
The fact that there is no option prevent any commit at all was a concious 
choice:

a) the processor is basically useless unless something does a commit -- 
there's no point in doing deletes every 30 seconds if we only want to 
bother having a new searcher every 60 seconds -- it just means we're doing 
twice the work w/o any added benefit.

b) a softCommit+openSeacher is a No-Op unless there is soemthing to actauly 
commit. (see SOLR-5783 and TestIndexSearcher.testReopen)

{quote}

...there is no value add in a new option to prevent the commit from being fired 
-- it won't do anything unless there is something to commit.

If DocExpirationUpdateProcessorFactory users want commits to happen less 
frequently, then they can change the autoDeletePeriodSeconds to be less 
frequent, because by definition there is no utility in a low 
autoDeletePeriodSeconds value unless searchers are being re-opened with the 
same frequency.



bq. maybe this... is a Jira issue? Pretty unclear, having the menu item titled 
"Create Service Desk Request" when you're told to create a Jira issue...

sigh.  yes you did in fact create a jira issue -- aparently jira was upgraded 
recently and the term "Issue" doesn't seem to exist anymore, they appear to 
have been renamed into "Service Desk Request" ... i don't know why.

> Disable expiration processor commits
> 
>
> Key: SOLR-9841
> URL: https://issues.apache.org/jira/browse/SOLR-9841
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: Brent Pearson
>
> http://lucene.472066.n3.nabble.com/Adding-DocExpirationUpdateProcessorFactory-causes-quot-Overlapping-onDeckSearchers-quot-warnings-td4309155.html
> Need a way to have document expiration update processor *not* trigger 
> additional commits.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9841) Disable expiration processor commits


 [ 
https://issues.apache.org/jira/browse/SOLR-9841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brent Pearson updated SOLR-9841:

Description: 
http://lucene.472066.n3.nabble.com/Adding-DocExpirationUpdateProcessorFactory-causes-quot-Overlapping-onDeckSearchers-quot-warnings-td4309155.html

Need a way to have document expiration update processor *not* trigger 
additional commits.

  was:hmmm... maybe this... is a Jira issue? Pretty unclear, having the menu 
item titled "Create Service Desk Request" when you're told to create a Jira 
issue...


> Disable expiration processor commits
> 
>
> Key: SOLR-9841
> URL: https://issues.apache.org/jira/browse/SOLR-9841
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: Brent Pearson
>
> http://lucene.472066.n3.nabble.com/Adding-DocExpirationUpdateProcessorFactory-causes-quot-Overlapping-onDeckSearchers-quot-warnings-td4309155.html
> Need a way to have document expiration update processor *not* trigger 
> additional commits.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9841) Create a Jira issue


[ 
https://issues.apache.org/jira/browse/SOLR-9841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736728#comment-15736728
 ] 

Brent Pearson commented on SOLR-9841:
-

hmmm... maybe this... is a Jira issue? Pretty unclear, having the menu item 
titled "Create Service Desk Request" when you're told to create a Jira issue...

> Create a Jira issue
> ---
>
> Key: SOLR-9841
> URL: https://issues.apache.org/jira/browse/SOLR-9841
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: Brent Pearson
>
> hmmm... maybe this... is a Jira issue? Pretty unclear, having the menu item 
> titled "Create Service Desk Request" when you're told to create a Jira 
> issue...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9841) Disable expiration processor commits


 [ 
https://issues.apache.org/jira/browse/SOLR-9841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brent Pearson updated SOLR-9841:

Summary: Disable expiration processor commits  (was: Create a Jira issue)

> Disable expiration processor commits
> 
>
> Key: SOLR-9841
> URL: https://issues.apache.org/jira/browse/SOLR-9841
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: Brent Pearson
>
> hmmm... maybe this... is a Jira issue? Pretty unclear, having the menu item 
> titled "Create Service Desk Request" when you're told to create a Jira 
> issue...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-MAVEN] Lucene-Solr-Maven-master #1903: POMs out of sync

Build: https://builds.apache.org/job/Lucene-Solr-Maven-master/1903/

No tests ran.

Build Log:
[...truncated 30807 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Maven-master/build.xml:783: The 
following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Maven-master/build.xml:324: The 
following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Maven-master/lucene/build.xml:423:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Maven-master/lucene/common-build.xml:2177:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Maven-master/lucene/common-build.xml:1667:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Maven-master/lucene/common-build.xml:566:
 Error deploying artifact 'org.apache.lucene:lucene-spatial-extras:jar': Error 
deploying artifact: Error transferring file

Total time: 14 minutes 1 second
Build step 'Invoke Ant' marked build as failure
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9841) Create a Jira issue


 [ 
https://issues.apache.org/jira/browse/SOLR-9841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brent Pearson updated SOLR-9841:

Description: hmmm... maybe this... is a Jira issue? Pretty unclear, having 
the menu item titled "Create Service Desk Request" when you're told to create a 
Jira issue...

> Create a Jira issue
> ---
>
> Key: SOLR-9841
> URL: https://issues.apache.org/jira/browse/SOLR-9841
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: Brent Pearson
>
> hmmm... maybe this... is a Jira issue? Pretty unclear, having the menu item 
> titled "Create Service Desk Request" when you're told to create a Jira 
> issue...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9841) Create a Jira issue


[ 
https://issues.apache.org/jira/browse/SOLR-9841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736723#comment-15736723
 ] 

Brent Pearson commented on SOLR-9841:
-

http://lucene.472066.n3.nabble.com/Adding-DocExpirationUpdateProcessorFactory-causes-quot-Overlapping-onDeckSearchers-quot-warnings-td4309155.html

Need to create a Jira issue, so I can submit a patch to disable commits from 
the doc expiration update processor.

> Create a Jira issue
> ---
>
> Key: SOLR-9841
> URL: https://issues.apache.org/jira/browse/SOLR-9841
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: Brent Pearson
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9841) Create a Jira issue

2016-12-09 Thread Anonymous (JIRA)

Anonymous created SOLR-9841:
---

 Summary: Create a Jira issue
 Key: SOLR-9841
 URL: https://issues.apache.org/jira/browse/SOLR-9841
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
  Components: SolrCloud
Reporter: Brent Pearson






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-7570) Tragic events during merges can lead to deadlock

2016-12-09 Thread ASF subversion and git services (JIRA)


 [ 
https://issues.apache.org/jira/browse/LUCENE-7570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael McCandless resolved LUCENE-7570.

   Resolution: Fixed
Fix Version/s: 6.4
   master (7.0)

> Tragic events during merges can lead to deadlock
> 
>
> Key: LUCENE-7570
> URL: https://issues.apache.org/jira/browse/LUCENE-7570
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index
>Affects Versions: 5.5, master (7.0)
>Reporter: Joey Echeverria
> Fix For: master (7.0), 6.4
>
> Attachments: LUCENE-7570.patch, thread_dump.txt
>
>
> When an {{IndexWriter#commit()}} is stalled due to too many pending merges, 
> you can get a deadlock if the currently active merge thread hits a tragic 
> event.
> # The thread performing the commit synchronizes on the the {{commitLock}} in 
> {{commitInternal}}.
> # The thread goes on to to call {{ConcurrentMergeScheduler#doStall()}} which 
> {{waits()}} on the {{ConcurrentMergeScheduler}} object. This release the 
> merge scheduler's monitor lock, but not the {{commitLock}} in {{IndexWriter}}.
> # Sometime after this wait begins, the merge thread gets a tragic exception 
> can calls {{IndexWriter#tragicEvent()}} which in turn calls 
> {{IndexWriter#rollbackInternal()}}.
> # The {{IndexWriter#rollbackInternal()}} synchronizes on the {{commitLock}} 
> which is still held by the committing thread from (1) above which is waiting 
> on the merge(s) to complete. Hence, deadlock.
> We hit this bug with Lucene 5.5, but I looked at the code in the master 
> branch and it looks like the deadlock still exists there as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7570) Tragic events during merges can lead to deadlock


[ 
https://issues.apache.org/jira/browse/LUCENE-7570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736698#comment-15736698
 ] 

ASF subversion and git services commented on LUCENE-7570:
-

Commit ea3f8363319955c589eb3a7df59a031621852d3e in lucene-solr's branch 
refs/heads/branch_6x from Mike McCandless
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=ea3f836 ]

LUCENE-7570: don't run merges while holding the commitLock to prevent deadlock 
when merges are stalled and a tragic merge exception strikes


> Tragic events during merges can lead to deadlock
> 
>
> Key: LUCENE-7570
> URL: https://issues.apache.org/jira/browse/LUCENE-7570
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index
>Affects Versions: 5.5, master (7.0)
>Reporter: Joey Echeverria
> Fix For: master (7.0), 6.4
>
> Attachments: LUCENE-7570.patch, thread_dump.txt
>
>
> When an {{IndexWriter#commit()}} is stalled due to too many pending merges, 
> you can get a deadlock if the currently active merge thread hits a tragic 
> event.
> # The thread performing the commit synchronizes on the the {{commitLock}} in 
> {{commitInternal}}.
> # The thread goes on to to call {{ConcurrentMergeScheduler#doStall()}} which 
> {{waits()}} on the {{ConcurrentMergeScheduler}} object. This release the 
> merge scheduler's monitor lock, but not the {{commitLock}} in {{IndexWriter}}.
> # Sometime after this wait begins, the merge thread gets a tragic exception 
> can calls {{IndexWriter#tragicEvent()}} which in turn calls 
> {{IndexWriter#rollbackInternal()}}.
> # The {{IndexWriter#rollbackInternal()}} synchronizes on the {{commitLock}} 
> which is still held by the committing thread from (1) above which is waiting 
> on the merge(s) to complete. Hence, deadlock.
> We hit this bug with Lucene 5.5, but I looked at the code in the master 
> branch and it looks like the deadlock still exists there as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7570) Tragic events during merges can lead to deadlock

2016-12-09 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-7570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736693#comment-15736693
 ] 

ASF subversion and git services commented on LUCENE-7570:
-

Commit 2b073a2f296289617bea8256d7efec06049df739 in lucene-solr's branch 
refs/heads/master from Mike McCandless
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=2b073a2 ]

LUCENE-7570: don't run merges while holding the commitLock to prevent deadlock 
when merges are stalled and a tragic merge exception strikes


> Tragic events during merges can lead to deadlock
> 
>
> Key: LUCENE-7570
> URL: https://issues.apache.org/jira/browse/LUCENE-7570
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index
>Affects Versions: 5.5, master (7.0)
>Reporter: Joey Echeverria
> Attachments: LUCENE-7570.patch, thread_dump.txt
>
>
> When an {{IndexWriter#commit()}} is stalled due to too many pending merges, 
> you can get a deadlock if the currently active merge thread hits a tragic 
> event.
> # The thread performing the commit synchronizes on the the {{commitLock}} in 
> {{commitInternal}}.
> # The thread goes on to to call {{ConcurrentMergeScheduler#doStall()}} which 
> {{waits()}} on the {{ConcurrentMergeScheduler}} object. This release the 
> merge scheduler's monitor lock, but not the {{commitLock}} in {{IndexWriter}}.
> # Sometime after this wait begins, the merge thread gets a tragic exception 
> can calls {{IndexWriter#tragicEvent()}} which in turn calls 
> {{IndexWriter#rollbackInternal()}}.
> # The {{IndexWriter#rollbackInternal()}} synchronizes on the {{commitLock}} 
> which is still held by the committing thread from (1) above which is waiting 
> on the merge(s) to complete. Hence, deadlock.
> We hit this bug with Lucene 5.5, but I looked at the code in the master 
> branch and it looks like the deadlock still exists there as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

RE: lucene-solr:master: LUCENE-7581: don't allow updating a doc values field if it's used in the index sort

Hi,


> > +this.indexSortFields = Arrays.stream(sort.getSort()).map((s) ->
> > s.getField()).collect(Collectors.toSet());
> 
> I'd use method references instead of Lambdas:
> 
> Instead of: map((s) -> s.getField())
> Use: map(s::getField)

Correction: map(SortField::getField)

> This spares a synthetic lambda$xxx method that clutters stack trace and is
> more readable.
> 
> Uwe
> 
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: lucene-solr:master: LUCENE-7581: don't allow updating a doc values field if it's used in the index sort

2016-12-09 Thread Michael McCandless

Thanks Uwe, I'll fix.

Mike McCandless

http://blog.mikemccandless.com


On Fri, Dec 9, 2016 at 6:25 PM, Uwe Schindler  wrote:
> Hi,
>
>> +this.indexSortFields = Arrays.stream(sort.getSort()).map((s) ->
>> s.getField()).collect(Collectors.toSet());
>
> I'd use method references instead of Lambdas:
>
> Instead of: map((s) -> s.getField())
> Use: map(s::getField)
>
> This spares a synthetic lambda$xxx method that clutters stack trace and is 
> more readable.
>
> Uwe
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

RE: lucene-solr:master: LUCENE-7581: don't allow updating a doc values field if it's used in the index sort

Hi,

> +this.indexSortFields = Arrays.stream(sort.getSort()).map((s) ->
> s.getField()).collect(Collectors.toSet());

I'd use method references instead of Lambdas:

Instead of: map((s) -> s.getField())
Use: map(s::getField)

This spares a synthetic lambda$xxx method that clutters stack trace and is more 
readable.

Uwe


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-7581) IndexWriter#updateDocValues can break index sorting

2016-12-09 Thread ASF subversion and git services (JIRA)


 [ 
https://issues.apache.org/jira/browse/LUCENE-7581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael McCandless resolved LUCENE-7581.

   Resolution: Fixed
Fix Version/s: 6.4
   master (7.0)

Thank you [~jim.ferenczi]!

> IndexWriter#updateDocValues can break index sorting
> ---
>
> Key: LUCENE-7581
> URL: https://issues.apache.org/jira/browse/LUCENE-7581
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Ferenczi Jim
> Fix For: master (7.0), 6.4
>
> Attachments: LUCENE-7581.patch, LUCENE-7581.patch
>
>
> IndexWriter#updateDocValues can break index sorting if it is called on a 
> field that is used in the index sorting specification. 
> TestIndexSorting has a test for this case: #testConcurrentDVUpdates 
> but only L1 merge are checked. Any LN merge would fail the test because the 
> inner sort of the segment is not re-compute during/after DV updates.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7581) IndexWriter#updateDocValues can break index sorting


[ 
https://issues.apache.org/jira/browse/LUCENE-7581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736620#comment-15736620
 ] 

ASF subversion and git services commented on LUCENE-7581:
-

Commit 928fa91c894867f0543432e5036bb09615a6d7f1 in lucene-solr's branch 
refs/heads/branch_6x from Mike McCandless
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=928fa91 ]

LUCENE-7581: don't allow updating a doc values field if it's used in the index 
sort


> IndexWriter#updateDocValues can break index sorting
> ---
>
> Key: LUCENE-7581
> URL: https://issues.apache.org/jira/browse/LUCENE-7581
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Ferenczi Jim
> Attachments: LUCENE-7581.patch, LUCENE-7581.patch
>
>
> IndexWriter#updateDocValues can break index sorting if it is called on a 
> field that is used in the index sorting specification. 
> TestIndexSorting has a test for this case: #testConcurrentDVUpdates 
> but only L1 merge are checked. Any LN merge would fail the test because the 
> inner sort of the segment is not re-compute during/after DV updates.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7581) IndexWriter#updateDocValues can break index sorting

2016-12-09 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-7581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736610#comment-15736610
 ] 

ASF subversion and git services commented on LUCENE-7581:
-

Commit 4efbde4e76277f364952866c071bb953ca2be070 in lucene-solr's branch 
refs/heads/master from Mike McCandless
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=4efbde4 ]

LUCENE-7581: don't allow updating a doc values field if it's used in the index 
sort


> IndexWriter#updateDocValues can break index sorting
> ---
>
> Key: LUCENE-7581
> URL: https://issues.apache.org/jira/browse/LUCENE-7581
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Ferenczi Jim
> Attachments: LUCENE-7581.patch, LUCENE-7581.patch
>
>
> IndexWriter#updateDocValues can break index sorting if it is called on a 
> field that is used in the index sorting specification. 
> TestIndexSorting has a test for this case: #testConcurrentDVUpdates 
> but only L1 merge are checked. Any LN merge would fail the test because the 
> inner sort of the segment is not re-compute during/after DV updates.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7575) UnifiedHighlighter: add requireFieldMatch=false support


[ 
https://issues.apache.org/jira/browse/LUCENE-7575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736587#comment-15736587
 ] 

Michael McCandless commented on LUCENE-7575:


[~dsmiley] Please see [~hossman]'s comment above?

> UnifiedHighlighter: add requireFieldMatch=false support
> ---
>
> Key: LUCENE-7575
> URL: https://issues.apache.org/jira/browse/LUCENE-7575
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/highlighter
>Reporter: David Smiley
>Assignee: David Smiley
> Fix For: 6.4
>
> Attachments: LUCENE-7575.patch, LUCENE-7575.patch, LUCENE-7575.patch
>
>
> The UnifiedHighlighter (like the PostingsHighlighter) only supports 
> highlighting queries for the same fields that are being highlighted.  The 
> original Highlighter and FVH support loosening this, AKA 
> requireFieldMatch=false.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7584) Potential leak issue


[ 
https://issues.apache.org/jira/browse/LUCENE-7584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736581#comment-15736581
 ] 

Michael McCandless commented on LUCENE-7584:


Hmm which version of Lucene is this?  It looks quite old ... newer versions of 
Lucene put compound file writing under codec control.

Also, I don't think there's a leak in the above code, because if 
{{createOutput}} throws any exception, it must not leave any file handles open; 
this is part of its contract.

> Potential leak issue
> 
>
> Key: LUCENE-7584
> URL: https://issues.apache.org/jira/browse/LUCENE-7584
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Jaechang Nam
>Priority: Trivial
>
> May the following method have a potential leak like LUCENE-3251 when  
> directory.createOutput(fileName) throws IOException?
> (Found the code from the recent code snapshot)
> lucene/src/java/org/apache/lucene/index/CompoundFileWriter.java
> {noformat}
>public void close() throws IOException {
> if (merged)
> throw new IllegalStateException("Merge already performed");
> if (entries.isEmpty())
> throw new IllegalStateException("No entries to merge have been 
> defined");
> merged = true;
> // open the compound stream
> IndexOutput os = directory.createOutput(fileName);
> IOException priorException = null;
> try {
> // Write the Version info - must be a VInt because CFR reads a 
> VInt
> // in older versions!
> os.writeVInt(FORMAT_CURRENT);
> // Write the number of entries
> os.writeVInt(entries.size());
> // Write the directory with all offsets at 0.
> // Remember the positions of directory entries so that we can
> // adjust the offsets later
> long totalSize = 0;
> for (FileEntry fe : entries) {
> fe.directoryOffset = os.getFilePointer();
> os.writeLong(0);// for now
> os.writeString(IndexFileNames.stripSegmentName(fe.file));
> totalSize += directory.fileLength(fe.file);
> }
> // Pre-allocate size of file as optimization --
> ...
> } catch (IOException e) {
>   priorException = e;
> } finally {
>   IOUtils.closeSafely(priorException, os);
> }
> }
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7584) Potential leak issue


 [ 
https://issues.apache.org/jira/browse/LUCENE-7584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael McCandless updated LUCENE-7584:
---
Description: 
May the following method have a potential leak like LUCENE-3251 when  
directory.createOutput(fileName) throws IOException?

(Found the code from the recent code snapshot)

lucene/src/java/org/apache/lucene/index/CompoundFileWriter.java
{noformat}
   public void close() throws IOException {
if (merged)
throw new IllegalStateException("Merge already performed");

if (entries.isEmpty())
throw new IllegalStateException("No entries to merge have been 
defined");

merged = true;

// open the compound stream
IndexOutput os = directory.createOutput(fileName);
IOException priorException = null;
try {
// Write the Version info - must be a VInt because CFR reads a VInt
// in older versions!
os.writeVInt(FORMAT_CURRENT);

// Write the number of entries
os.writeVInt(entries.size());

// Write the directory with all offsets at 0.
// Remember the positions of directory entries so that we can
// adjust the offsets later
long totalSize = 0;
for (FileEntry fe : entries) {
fe.directoryOffset = os.getFilePointer();
os.writeLong(0);// for now
os.writeString(IndexFileNames.stripSegmentName(fe.file));
totalSize += directory.fileLength(fe.file);
}

// Pre-allocate size of file as optimization --
...
} catch (IOException e) {
  priorException = e;
} finally {
  IOUtils.closeSafely(priorException, os);
}
}
{noformat}

  was:
May the following method have a potential leak like LUCENE-3251 when  
directory.createOutput(fileName) throws IOException?

(Found the code from the recent code snapshot)

lucene/src/java/org/apache/lucene/index/CompoundFileWriter.java

   public void close() throws IOException {
if (merged)
throw new IllegalStateException("Merge already performed");

if (entries.isEmpty())
throw new IllegalStateException("No entries to merge have been 
defined");

merged = true;

// open the compound stream
IndexOutput os = directory.createOutput(fileName);
IOException priorException = null;
try {
// Write the Version info - must be a VInt because CFR reads a VInt
// in older versions!
os.writeVInt(FORMAT_CURRENT);

// Write the number of entries
os.writeVInt(entries.size());

// Write the directory with all offsets at 0.
// Remember the positions of directory entries so that we can
// adjust the offsets later
long totalSize = 0;
for (FileEntry fe : entries) {
fe.directoryOffset = os.getFilePointer();
os.writeLong(0);// for now
os.writeString(IndexFileNames.stripSegmentName(fe.file));
totalSize += directory.fileLength(fe.file);
}

// Pre-allocate size of file as optimization --
...
} catch (IOException e) {
  priorException = e;
} finally {
  IOUtils.closeSafely(priorException, os);
}
}



> Potential leak issue
> 
>
> Key: LUCENE-7584
> URL: https://issues.apache.org/jira/browse/LUCENE-7584
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Jaechang Nam
>Priority: Trivial
>
> May the following method have a potential leak like LUCENE-3251 when  
> directory.createOutput(fileName) throws IOException?
> (Found the code from the recent code snapshot)
> lucene/src/java/org/apache/lucene/index/CompoundFileWriter.java
> {noformat}
>public void close() throws IOException {
> if (merged)
> throw new IllegalStateException("Merge already performed");
> if (entries.isEmpty())
> throw new IllegalStateException("No entries to merge have been 
> defined");
> merged = true;
> // open the compound stream
> IndexOutput os = directory.createOutput(fileName);
> IOException priorException = null;
> try {
> // Write the Version info - must be a VInt because CFR reads a 
> VInt
> // in older versions!
> os.writeVInt(FORMAT_CURRENT);
> // Write the number of entries
> os.writeVInt(entries.size());
> // Write the directory with all offsets at 0.
> // Remember the positions of directory entries so that we can
> // adjust the offsets later
> long totalSize = 0;
> for

[jira] [Commented] (LUCENE-6989) Implement MMapDirectory unmapping for coming Java 9 changes


[ 
https://issues.apache.org/jira/browse/LUCENE-6989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736570#comment-15736570
 ] 

Uwe Schindler commented on LUCENE-6989:
---

I opened thread: 
http://mail.openjdk.java.net/pipermail/jigsaw-dev/2016-December/010454.html

> Implement MMapDirectory unmapping for coming Java 9 changes
> ---
>
> Key: LUCENE-6989
> URL: https://issues.apache.org/jira/browse/LUCENE-6989
> Project: Lucene - Core
>  Issue Type: Task
>  Components: core/store
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
>  Labels: Java9
> Fix For: 6.0
>
> Attachments: LUCENE-6989-disable5x.patch, 
> LUCENE-6989-disable5x.patch, LUCENE-6989-fixbuild148.patch, 
> LUCENE-6989-v2.patch, LUCENE-6989.patch, LUCENE-6989.patch, 
> LUCENE-6989.patch, LUCENE-6989.patch
>
>
> Originally, the sun.misc.Cleaner interface was declared as "critical API" in 
> [JEP 260|http://openjdk.java.net/jeps/260 ]
> Unfortunately the decission was changed in favor of a oficially supported 
> {{java.lang.ref.Cleaner}} API. A side effect of this change is to move all 
> existing {{sun.misc.Cleaner}} APIs into a non-exported package. This causes 
> our forceful unmapping to no longer work, because we can get the cleaner 
> instance via reflection, but trying to invoke it will throw one of the new 
> Jigsaw RuntimeException because it is completely inaccessible. This will make 
> our forceful unmapping fail. There are also no changes in Garbage collector, 
> the problem still exists.
> For more information see this [mailing list 
> thread|http://mail.openjdk.java.net/pipermail/core-libs-dev/2016-January/thread.html#38243].
> This commit will likely be done, making our unmapping efforts no longer 
> working. Alan Bateman is aware of this issue and will open a new issue at 
> OpenJDK to allow forceful unmapping without using the now private 
> sun.misc.Cleaner. The idea is to let the internal class sun.misc.Cleaner 
> implement the Runable interface, so we can simply cast to runable and call 
> the run() method to unmap. The code would then work. This will lead to minor 
> changes in our unmapper in MMapDirectory: An instanceof check and casting if 
> possible.
> I opened this issue to keep track and implement the changes as soon as 
> possible, so people will have working unmapping when java 9 comes out. 
> Current Lucene versions will no longer work with Java 9.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 999 - Unstable!

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/999/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.client.solrj.io.stream.StreamExpressionTest

Error Message:
1 thread leaked from SUITE scope at 
org.apache.solr.client.solrj.io.stream.StreamExpressionTest: 1) 
Thread[id=5100, 
name=OverseerHdfsCoreFailoverThread-97079926104588298-127.0.0.1:57819_solr-n_02,
 state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.] at 
java.lang.Thread.sleep(Native Method) at 
org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137)
 at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE 
scope at org.apache.solr.client.solrj.io.stream.StreamExpressionTest: 
   1) Thread[id=5100, 
name=OverseerHdfsCoreFailoverThread-97079926104588298-127.0.0.1:57819_solr-n_02,
 state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.]
at java.lang.Thread.sleep(Native Method)
at 
org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([760B05BA4FC141E1]:0)




Build Log:
[...truncated 13373 lines...]
   [junit4] Suite: org.apache.solr.client.solrj.io.stream.StreamExpressionTest
   [junit4]   2> Creating dataDir: 
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.io.stream.StreamExpressionTest_760B05BA4FC141E1-001/init-core-data-001
   [junit4]   2> 124551 INFO  
(SUITE-StreamExpressionTest-seed#[760B05BA4FC141E1]-worker) [] 
o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: 
@org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 124551 INFO  
(SUITE-StreamExpressionTest-seed#[760B05BA4FC141E1]-worker) [] 
o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in 
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.io.stream.StreamExpressionTest_760B05BA4FC141E1-001/tempDir-001
   [junit4]   2> 124552 INFO  
(SUITE-StreamExpressionTest-seed#[760B05BA4FC141E1]-worker) [] 
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 124552 INFO  (Thread-298) [] o.a.s.c.ZkTestServer client 
port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 124552 INFO  (Thread-298) [] o.a.s.c.ZkTestServer Starting 
server
   [junit4]   2> 124654 INFO  
(SUITE-StreamExpressionTest-seed#[760B05BA4FC141E1]-worker) [] 
o.a.s.c.ZkTestServer start zk server on port:57409
   [junit4]   2> 124668 INFO  (jetty-launcher-267-thread-2) [] 
o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 124668 INFO  (jetty-launcher-267-thread-3) [] 
o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 124669 INFO  (jetty-launcher-267-thread-1) [] 
o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 124670 INFO  (jetty-launcher-267-thread-1) [] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@4952160c{/solr,null,AVAILABLE}
   [junit4]   2> 124671 INFO  (jetty-launcher-267-thread-2) [] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@60862ca7{/solr,null,AVAILABLE}
   [junit4]   2> 124671 INFO  (jetty-launcher-267-thread-2) [] 
o.e.j.s.AbstractConnector Started 
ServerConnector@6cc63e71{HTTP/1.1,[http/1.1]}{127.0.0.1:57819}
   [junit4]   2> 124671 INFO  (jetty-launcher-267-thread-2) [] 
o.e.j.s.Server Started @127442ms
   [junit4]   2> 124671 INFO  (jetty-launcher-267-thread-2) [] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=57819}
   [junit4]   2> 124672 ERROR (jetty-launcher-267-thread-2) [] 
o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 124678 INFO  (jetty-launcher-267-thread-4) [] 
o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 124679 INFO  (jetty-launcher-267-thread-2) [] 
o.a.s.s.SolrDispatchFilter  ___  _   Welcome to Apache Solr? version 
7.0.0
   [junit4]   2> 124679 INFO  (jetty-launcher-267-thread-2) [] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 124679 INFO  (jetty-launcher-267-thread-2) [] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 124678 INFO  (jetty-launcher-267-thread-3) [] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@145d76b6{/solr,null,AVAILABLE}
   [junit4]   2> 124679 INFO  (jetty-launcher-267-thread-2) [] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|Start time: 
2016-12-09T22:21:54.751Z
   [junit4]   2> 124680 INFO  (jetty-launcher-267-thread-3) [] 
o.e.j.s.AbstractConnector Started

[jira] [Commented] (LUCENE-7581) IndexWriter#updateDocValues can break index sorting


[ 
https://issues.apache.org/jira/browse/LUCENE-7581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736549#comment-15736549
 ] 

Michael McCandless commented on LUCENE-7581:


Thanks [~jim.ferenczi], this looks great; I'll push shortly.

> IndexWriter#updateDocValues can break index sorting
> ---
>
> Key: LUCENE-7581
> URL: https://issues.apache.org/jira/browse/LUCENE-7581
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Ferenczi Jim
> Attachments: LUCENE-7581.patch, LUCENE-7581.patch
>
>
> IndexWriter#updateDocValues can break index sorting if it is called on a 
> field that is used in the index sorting specification. 
> TestIndexSorting has a test for this case: #testConcurrentDVUpdates 
> but only L1 merge are checked. Any LN merge would fail the test because the 
> inner sort of the segment is not re-compute during/after DV updates.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-EA] Lucene-Solr-master-Linux (64bit/jdk-9-ea+147) - Build # 18489 - Still unstable!

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/18489/
Java: 64bit/jdk-9-ea+147 -XX:+UseCompressedOops -XX:+UseG1GC

1 tests failed.
FAILED:  org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test

Error Message:
Expected 2 of 3 replicas to be active but only found 1; 
[core_node3:{"core":"c8n_1x3_lf_shard1_replica3","base_url":"http://127.0.0.1:34242/euzvh/kq","node_name":"127.0.0.1:34242_euzvh%2Fkq","state":"active","leader":"true"}];
 clusterState: DocCollection(c8n_1x3_lf//clusterstate.json/30)={   
"replicationFactor":"3",   "shards":{"shard1":{   
"range":"8000-7fff",   "state":"active",   "replicas":{ 
"core_node1":{   "state":"down",   
"base_url":"http://127.0.0.1:39111/euzvh/kq;,   
"core":"c8n_1x3_lf_shard1_replica1",   
"node_name":"127.0.0.1:39111_euzvh%2Fkq"}, "core_node2":{   
"core":"c8n_1x3_lf_shard1_replica2",   
"base_url":"http://127.0.0.1:42583/euzvh/kq;,   
"node_name":"127.0.0.1:42583_euzvh%2Fkq",   "state":"down"}, 
"core_node3":{   "core":"c8n_1x3_lf_shard1_replica3",   
"base_url":"http://127.0.0.1:34242/euzvh/kq;,   
"node_name":"127.0.0.1:34242_euzvh%2Fkq",   "state":"active",   
"leader":"true",   "router":{"name":"compositeId"},   
"maxShardsPerNode":"1",   "autoAddReplicas":"false"}

Stack Trace:
java.lang.AssertionError: Expected 2 of 3 replicas to be active but only found 
1; 
[core_node3:{"core":"c8n_1x3_lf_shard1_replica3","base_url":"http://127.0.0.1:34242/euzvh/kq","node_name":"127.0.0.1:34242_euzvh%2Fkq","state":"active","leader":"true"}];
 clusterState: DocCollection(c8n_1x3_lf//clusterstate.json/30)={
  "replicationFactor":"3",
  "shards":{"shard1":{
  "range":"8000-7fff",
  "state":"active",
  "replicas":{
"core_node1":{
  "state":"down",
  "base_url":"http://127.0.0.1:39111/euzvh/kq;,
  "core":"c8n_1x3_lf_shard1_replica1",
  "node_name":"127.0.0.1:39111_euzvh%2Fkq"},
"core_node2":{
  "core":"c8n_1x3_lf_shard1_replica2",
  "base_url":"http://127.0.0.1:42583/euzvh/kq;,
  "node_name":"127.0.0.1:42583_euzvh%2Fkq",
  "state":"down"},
"core_node3":{
  "core":"c8n_1x3_lf_shard1_replica3",
  "base_url":"http://127.0.0.1:34242/euzvh/kq;,
  "node_name":"127.0.0.1:34242_euzvh%2Fkq",
  "state":"active",
  "leader":"true",
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"1",
  "autoAddReplicas":"false"}
at 
__randomizedtesting.SeedInfo.seed([7EAF7429462E0E2D:F6FB4BF3E8D263D5]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.testRf3WithLeaderFailover(LeaderFailoverAfterPartitionTest.java:170)
at 
org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test(LeaderFailoverAfterPartitionTest.java:57)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:538)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at

[jira] [Commented] (LUCENE-7585) Interface for common parameters used across analysis factories

2016-12-09 Thread Ahmet Arslan (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-7585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736501#comment-15736501
 ] 

Ahmet Arslan commented on LUCENE-7585:
--

Thank you for looking into this. Initially, I was planning to move all existing 
parameters to a common interface.
I figured that the interface will grow very large since certain factories have 
many specific parameters. 
I moved the most common parameters to the interface. However, there still 
remains a lot in the codebase.
For example, ngram package has "minGramSize" and "maxGramSize" in common. 
Phonetic module has "maxCodeLength" and "inject."

What could be the preferred course of action here?

* Handle packages and modules locally? If yes how?
* Move all parameters to the interface unconditionally.
* Devise an algorithm: Move if a parameter is shared by at least two package or 
module.
* ?

> Interface for common parameters used across analysis factories
> --
>
> Key: LUCENE-7585
> URL: https://issues.apache.org/jira/browse/LUCENE-7585
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/analysis
>Affects Versions: 6.3
>Reporter: Ahmet Arslan
>Assignee: David Smiley
>Priority: Minor
> Fix For: master (7.0)
>
> Attachments: LUCENE-7585.patch, LUCENE-7585.patch
>
>
> Certain parameters (String constants) are same/common for multiple analysis 
> factories. Some examples are {{ignoreCase}}, {{dictionary}}, and 
> {{preserveOriginal}}. These string constants are handled inconsistently in 
> different factories. This is an effort to define most common constants in 
> ({{CommonAnalysisFactoryParams}}) interface and reuse them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9840) Add a unit test for LDAP integration

2016-12-09 Thread Hrishikesh Gadre (JIRA)

Hrishikesh Gadre created SOLR-9840:
--

 Summary: Add a unit test for LDAP integration
 Key: SOLR-9840
 URL: https://issues.apache.org/jira/browse/SOLR-9840
 Project: Solr
  Issue Type: Sub-task
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Hrishikesh Gadre
Priority: Minor


SOLR-9513 introduced a generic Hadoop authentication plugin which can be used 
to configure LDAP authentication functionality in Hadoop. This jira is to track 
the work required for adding a unit test for LDAP integration.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7585) Interface for common parameters used across analysis factories

2016-12-09 Thread Ahmet Arslan (JIRA)


 [ 
https://issues.apache.org/jira/browse/LUCENE-7585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ahmet Arslan updated LUCENE-7585:
-
Attachment: LUCENE-7585.patch

Properly created patch that includes proposed changes (alphabetisation and 
lucene_match_version). Ant {{documentation-lint}} complains about factories  of 
icu. Any pointer how to fix it?

> Interface for common parameters used across analysis factories
> --
>
> Key: LUCENE-7585
> URL: https://issues.apache.org/jira/browse/LUCENE-7585
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/analysis
>Affects Versions: 6.3
>Reporter: Ahmet Arslan
>Assignee: David Smiley
>Priority: Minor
> Fix For: master (7.0)
>
> Attachments: LUCENE-7585.patch, LUCENE-7585.patch
>
>
> Certain parameters (String constants) are same/common for multiple analysis 
> factories. Some examples are {{ignoreCase}}, {{dictionary}}, and 
> {{preserveOriginal}}. These string constants are handled inconsistently in 
> different factories. This is an effort to define most common constants in 
> ({{CommonAnalysisFactoryParams}}) interface and reuse them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-MAVEN] Lucene-Solr-Maven-6.x #206: POMs out of sync

Build: https://builds.apache.org/job/Lucene-Solr-Maven-6.x/206/

No tests ran.

Build Log:
[...truncated 34893 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Maven-6.x/build.xml:783: The 
following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Maven-6.x/build.xml:324: The 
following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Maven-6.x/solr/build.xml:691: 
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Maven-6.x/solr/common-build.xml:463:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Maven-6.x/lucene/common-build.xml:1667:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Maven-6.x/lucene/common-build.xml:566:
 Error deploying artifact 'org.apache.solr:solr-dataimporthandler:jar': Error 
installing artifact's metadata: Error while deploying metadata: Failed to 
transfer file: 
https://repository.apache.org/content/repositories/snapshots/org/apache/solr/solr-dataimporthandler/maven-metadata.xml.
 Return code is: 400

Total time: 34 minutes 44 seconds
Build step 'Invoke Ant' marked build as failure
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (LUCENE-6989) Implement MMapDirectory unmapping for coming Java 9 changes


[ 
https://issues.apache.org/jira/browse/LUCENE-6989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736398#comment-15736398
 ] 

Uwe Schindler edited comment on LUCENE-6989 at 12/9/16 9:29 PM:


For now I reverted back to build 147 of JDK 9 on Policeman Jenkins as the whole 
build system no longer works. I will check out with JDK guys how to proceed.

In addition to this mmap unmapping problem, RAMUsageEstimator in the test 
framework is also broken, and Groovy. Both seem to be affected by the same 
issue: setAccessible is completely forbidden, without any special cases that 
are allowed (see https://bugs.openjdk.java.net/browse/JDK-8148117 and 
https://bugs.openjdk.java.net/browse/JDK-8132928 for more details). I think 
that's a bug.


was (Author: thetaphi):
For now I reverted back to build 147 of JDK 9. I will check out with

In addition to this problem, RAMUsageEstimator in the test framework is also 
broken, and Groovy. Both seem to be affected by the same issue: setAccessible 
is completely forbidden, without any special cases that are allowed (see 
https://bugs.openjdk.java.net/browse/JDK-8148117 and 
https://bugs.openjdk.java.net/browse/JDK-8132928 for more details). I think 
that's a bug.

> Implement MMapDirectory unmapping for coming Java 9 changes
> ---
>
> Key: LUCENE-6989
> URL: https://issues.apache.org/jira/browse/LUCENE-6989
> Project: Lucene - Core
>  Issue Type: Task
>  Components: core/store
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
>  Labels: Java9
> Fix For: 6.0
>
> Attachments: LUCENE-6989-disable5x.patch, 
> LUCENE-6989-disable5x.patch, LUCENE-6989-fixbuild148.patch, 
> LUCENE-6989-v2.patch, LUCENE-6989.patch, LUCENE-6989.patch, 
> LUCENE-6989.patch, LUCENE-6989.patch
>
>
> Originally, the sun.misc.Cleaner interface was declared as "critical API" in 
> [JEP 260|http://openjdk.java.net/jeps/260 ]
> Unfortunately the decission was changed in favor of a oficially supported 
> {{java.lang.ref.Cleaner}} API. A side effect of this change is to move all 
> existing {{sun.misc.Cleaner}} APIs into a non-exported package. This causes 
> our forceful unmapping to no longer work, because we can get the cleaner 
> instance via reflection, but trying to invoke it will throw one of the new 
> Jigsaw RuntimeException because it is completely inaccessible. This will make 
> our forceful unmapping fail. There are also no changes in Garbage collector, 
> the problem still exists.
> For more information see this [mailing list 
> thread|http://mail.openjdk.java.net/pipermail/core-libs-dev/2016-January/thread.html#38243].
> This commit will likely be done, making our unmapping efforts no longer 
> working. Alan Bateman is aware of this issue and will open a new issue at 
> OpenJDK to allow forceful unmapping without using the now private 
> sun.misc.Cleaner. The idea is to let the internal class sun.misc.Cleaner 
> implement the Runable interface, so we can simply cast to runable and call 
> the run() method to unmap. The code would then work. This will lead to minor 
> changes in our unmapper in MMapDirectory: An instanceof check and casting if 
> possible.
> I opened this issue to keep track and implement the changes as soon as 
> possible, so people will have working unmapping when java 9 comes out. 
> Current Lucene versions will no longer work with Java 9.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6989) Implement MMapDirectory unmapping for coming Java 9 changes


[ 
https://issues.apache.org/jira/browse/LUCENE-6989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736398#comment-15736398
 ] 

Uwe Schindler commented on LUCENE-6989:
---

For now I reverted back to build 147 of JDK 9. I will check out with

In addition to this problem, RAMUsageEstimator in the test framework is also 
broken, and Groovy. Both seem to be affected by the same issue: setAccessible 
is completely forbidden, without any special cases that are allowed (see 
https://bugs.openjdk.java.net/browse/JDK-8148117 and 
https://bugs.openjdk.java.net/browse/JDK-8132928 for more details). I think 
that's a bug.

> Implement MMapDirectory unmapping for coming Java 9 changes
> ---
>
> Key: LUCENE-6989
> URL: https://issues.apache.org/jira/browse/LUCENE-6989
> Project: Lucene - Core
>  Issue Type: Task
>  Components: core/store
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
>  Labels: Java9
> Fix For: 6.0
>
> Attachments: LUCENE-6989-disable5x.patch, 
> LUCENE-6989-disable5x.patch, LUCENE-6989-fixbuild148.patch, 
> LUCENE-6989-v2.patch, LUCENE-6989.patch, LUCENE-6989.patch, 
> LUCENE-6989.patch, LUCENE-6989.patch
>
>
> Originally, the sun.misc.Cleaner interface was declared as "critical API" in 
> [JEP 260|http://openjdk.java.net/jeps/260 ]
> Unfortunately the decission was changed in favor of a oficially supported 
> {{java.lang.ref.Cleaner}} API. A side effect of this change is to move all 
> existing {{sun.misc.Cleaner}} APIs into a non-exported package. This causes 
> our forceful unmapping to no longer work, because we can get the cleaner 
> instance via reflection, but trying to invoke it will throw one of the new 
> Jigsaw RuntimeException because it is completely inaccessible. This will make 
> our forceful unmapping fail. There are also no changes in Garbage collector, 
> the problem still exists.
> For more information see this [mailing list 
> thread|http://mail.openjdk.java.net/pipermail/core-libs-dev/2016-January/thread.html#38243].
> This commit will likely be done, making our unmapping efforts no longer 
> working. Alan Bateman is aware of this issue and will open a new issue at 
> OpenJDK to allow forceful unmapping without using the now private 
> sun.misc.Cleaner. The idea is to let the internal class sun.misc.Cleaner 
> implement the Runable interface, so we can simply cast to runable and call 
> the run() method to unmap. The code would then work. This will lead to minor 
> changes in our unmapper in MMapDirectory: An instanceof check and casting if 
> possible.
> I opened this issue to keep track and implement the changes as soon as 
> possible, so people will have working unmapping when java 9 comes out. 
> Current Lucene versions will no longer work with Java 9.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9837) Performance regression of numeric field uninversion time

2016-12-09 Thread JIRA


[ 
https://issues.apache.org/jira/browse/SOLR-9837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736354#comment-15736354
 ] 

Tomás Fernández Löbbe commented on SOLR-9837:
-

That would be great. I always wanted something like that for Solr too, but I 
never had the time to work on it. 

> Performance regression of numeric field uninversion time
> 
>
> Key: SOLR-9837
> URL: https://issues.apache.org/jira/browse/SOLR-9837
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: master (7.0)
>Reporter: Yonik Seeley
>Assignee: Yonik Seeley
> Fix For: master (7.0)
>
>
> Somehow related to LUCENE-7407, after the transition, the uninvert time of 
> numeric fields has gone up substantially.  I haven't tested all field types 
> yet, just integer fields, which show a 55% performance regression for the 
> initial uninvert time.
> This was tested with a numeric field of cardinality 1M on a 10M doc index.
> {code}
> q=id:1=my_numeric_field desc
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 223 - Failure

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/223/

4 tests failed.
FAILED:  org.apache.lucene.index.TestDuelingCodecsAtNight.testBigEquals

Error Message:
this IndexWriter is closed

Stack Trace:
org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:753)
at org.apache.lucene.index.IndexWriter.getConfig(IndexWriter.java:1093)
at 
org.apache.lucene.index.RandomIndexWriter.maybeFlushOrCommit(RandomIndexWriter.java:180)
at 
org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:174)
at 
org.apache.lucene.index.TestDuelingCodecs.createRandomIndex(TestDuelingCodecs.java:139)
at 
org.apache.lucene.index.TestDuelingCodecsAtNight.testBigEquals(TestDuelingCodecsAtNight.java:34)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.OutOfMemoryError: Java heap space
at org.apache.lucene.util.fst.BytesStore.skipBytes(BytesStore.java:307)
at org.apache.lucene.util.fst.FST.addNode(FST.java:792)
at org.apache.lucene.util.fst.NodeHash.add(NodeHash.java:126)
at org.apache.lucene.util.fst.Builder.compileNode(Builder.java:214)
at org.apache.lucene.util.fst.Builder.freezeTail(Builder.java:310)
at

[JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+148) - Build # 18484 - Failure!

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/18484/
Java: 32bit/jdk-9-ea+148 -server -XX:+UseParallelGC

30 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.lucene.index.TestCodecs

Error Message:
Clean up static fields (in @AfterClass?) and null them, your test still has 
references to classes of which the sizes cannot be measured due to security 
restrictions or Java 9 module encapsulation:   - private static 
java.lang.String[] org.apache.lucene.index.TestCodecs.fieldNames

Stack Trace:
junit.framework.AssertionFailedError: Clean up static fields (in @AfterClass?) 
and null them, your test still has references to classes of which the sizes 
cannot be measured due to security restrictions or Java 9 module encapsulation:
  - private static java.lang.String[] 
org.apache.lucene.index.TestCodecs.fieldNames
at __randomizedtesting.SeedInfo.seed([15305BE129154701]:0)
at 
com.carrotsearch.randomizedtesting.rules.StaticFieldsInvariantRule$1.afterAlways(StaticFieldsInvariantRule.java:146)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.base/java.lang.Thread.run(Thread.java:844)
Caused by: java.lang.IllegalStateException: Unable to access 'private final 
byte[] java.lang.String.value' to estimate memory usage
at 
com.carrotsearch.randomizedtesting.rules.RamUsageEstimator.createCacheEntry(RamUsageEstimator.java:602)
at 
com.carrotsearch.randomizedtesting.rules.RamUsageEstimator.measureSizeOf(RamUsageEstimator.java:545)
at 
com.carrotsearch.randomizedtesting.rules.RamUsageEstimator.sizeOfAll(RamUsageEstimator.java:387)
at 
com.carrotsearch.randomizedtesting.rules.StaticFieldsInvariantRule$1.afterAlways(StaticFieldsInvariantRule.java:129)
... 10 more
Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make field 
private final byte[] java.lang.String.value accessible: module java.base does 
not "opens java.lang" to unnamed module @1a36217
at 
java.base/jdk.internal.reflect.Reflection.throwInaccessibleObjectException(Reflection.java:427)
at 
java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:201)
at 
java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:171)
at java.base/java.lang.reflect.Field.setAccessible(Field.java:165)
at 
com.carrotsearch.randomizedtesting.rules.RamUsageEstimator$3.run(RamUsageEstimator.java:597)
at 
com.carrotsearch.randomizedtesting.rules.RamUsageEstimator$3.run(RamUsageEstimator.java:594)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at 
com.carrotsearch.randomizedtesting.rules.RamUsageEstimator.createCacheEntry(RamUsageEstimator.java:594)
... 13 more


FAILED:  
junit.framework.TestSuite.org.apache.lucene.index.TestIndexWriterExceptions

Error Message:
Clean up static fields (in @AfterClass?) and null them, your test still has 
references to classes of which the sizes cannot be measured due to security 
restrictions or Java 9 module encapsulation:   - private static 
java.lang.String 
org.apache.lucene.index.TestIndexWriterExceptions.CRASH_FAIL_MESSAGE

Stack Trace:
junit.framework.AssertionFailedError: Clean up static fields (in @AfterClass?) 
and null them, your test still has references to classes of which the sizes 
cannot be measured due to security restrictions or Java 9 module encapsulation:
  - private static java.lang.String 
org.apache.lucene.index.TestIndexWriterExceptions.CRASH_FAIL_MESSAGE
at __randomizedtesting.SeedInfo.seed([15305BE129154701]:0)
at 
com.carrotsearch.randomizedtesting.rules.StaticFieldsInvariantRule$1.afterAlways(StaticFieldsInvariantRule.java:146)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at

[jira] [Comment Edited] (SOLR-9837) Performance regression of numeric field uninversion time

2016-12-09 Thread Shawn Heisey (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-9837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736092#comment-15736092
 ] 

Shawn Heisey edited comment on SOLR-9837 at 12/9/16 7:32 PM:
-

What would it take to do something like the following for Solr?

https://home.apache.org/~mikemccand/lucenebench/


was (Author: elyograg):
What would it take to do something like the following for Solr?


> Performance regression of numeric field uninversion time
> 
>
> Key: SOLR-9837
> URL: https://issues.apache.org/jira/browse/SOLR-9837
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: master (7.0)
>Reporter: Yonik Seeley
>Assignee: Yonik Seeley
> Fix For: master (7.0)
>
>
> Somehow related to LUCENE-7407, after the transition, the uninvert time of 
> numeric fields has gone up substantially.  I haven't tested all field types 
> yet, just integer fields, which show a 55% performance regression for the 
> initial uninvert time.
> This was tested with a numeric field of cardinality 1M on a 10M doc index.
> {code}
> q=id:1=my_numeric_field desc
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9837) Performance regression of numeric field uninversion time

2016-12-09 Thread Shawn Heisey (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-9837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736092#comment-15736092
 ] 

Shawn Heisey commented on SOLR-9837:


What would it take to do something like the following for Solr?


> Performance regression of numeric field uninversion time
> 
>
> Key: SOLR-9837
> URL: https://issues.apache.org/jira/browse/SOLR-9837
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: master (7.0)
>Reporter: Yonik Seeley
>Assignee: Yonik Seeley
> Fix For: master (7.0)
>
>
> Somehow related to LUCENE-7407, after the transition, the uninvert time of 
> numeric fields has gone up substantially.  I haven't tested all field types 
> yet, just integer fields, which show a 55% performance regression for the 
> initial uninvert time.
> This was tested with a numeric field of cardinality 1M on a 10M doc index.
> {code}
> q=id:1=my_numeric_field desc
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 3699 - Still Unstable!

2016-12-09 Thread ASF subversion and git services (JIRA)

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3699/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseSerialGC

1 tests failed.
FAILED:  
org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigAliasReplication

Error Message:
expected:<1> but was:<0>

Stack Trace:
java.lang.AssertionError: expected:<1> but was:<0>
at 
__randomizedtesting.SeedInfo.seed([A79BEAB82FF795F5:50E804E0E91F3A13]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at 
org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigAliasReplication(TestReplicationHandler.java:1329)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11667 lines...]
   [junit4] Suite:

[jira] [Commented] (SOLR-9835) Create another replication mode for SolrCloud

2016-12-09 Thread Pushkar Raste (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-9835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736051#comment-15736051
 ] 

Pushkar Raste commented on SOLR-9835:
-

Instead of periodic polling, can leader upon receiving and processing a commit 
command, send a notification to replicas asking them to sync up?

> Create another replication mode for SolrCloud
> -
>
> Key: SOLR-9835
> URL: https://issues.apache.org/jira/browse/SOLR-9835
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Cao Manh Dat
>
> The current replication mechanism of SolrCloud is called state machine, which 
> replicas start in same initial state and for each input, the input is 
> distributed across replicas so all replicas will end up with same next state. 
> But this type of replication have some drawbacks
> - The commit (which costly) have to run on all replicas
> - Slow recovery, because if replica miss more than N updates on its down 
> time, the replica have to download entire index from its leader.
> So we create create another replication mode for SolrCloud called state 
> transfer, which acts like master/slave replication. In basically
> - Leader distribute the update to other replicas, but the leader only apply 
> the update to IW, other replicas just store the update to UpdateLog (act like 
> replication).
> - Replicas frequently polling the latest segments from leader.
> Pros:
> - Lightweight for indexing, because only leader are running the commit, 
> updates.
> - Very fast recovery, replicas just have to download the missing segments.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9839) Use 'useFactory' in tests instead of setting manually

2016-12-09 Thread Mike Drob (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-9839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736019#comment-15736019
 ] 

Mike Drob commented on SOLR-9839:
-

Sorry for being unclear. The patch file I posted would automatically include a 
commit message if applied using {{git am}}. That commit message was the one I 
meant.

> Use 'useFactory' in tests instead of setting manually
> -
>
> Key: SOLR-9839
> URL: https://issues.apache.org/jira/browse/SOLR-9839
> Project: Solr
>  Issue Type: Test
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Tests
>Reporter: Mike Drob
>Priority: Minor
> Attachments: SOLR-9839.patch
>
>
> We have several tests that will explicitly set a directory factory via 
> SysProp, some of which forget to unset it.
> We should use {{useFactory}} so that we can benefit from the call to 
> {{resetFactory}} in {{SolrTestCaseJ4.teardownTestCases}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9819) Upgrade commons-fileupload to 1.3.2


[ 
https://issues.apache.org/jira/browse/SOLR-9819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736002#comment-15736002
 ] 

ASF subversion and git services commented on SOLR-9819:
---

Commit 5fb4be2aa3a36a8ebf15dc2a77c9d00b10104760 in lucene-solr's branch 
refs/heads/branch_5_5 from anshum
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=5fb4be2 ]

SOLR-9819: Fix solr/CHANGES.txt for 5.5.4


> Upgrade commons-fileupload to 1.3.2
> ---
>
> Key: SOLR-9819
> URL: https://issues.apache.org/jira/browse/SOLR-9819
> Project: Solr
>  Issue Type: Improvement
>  Components: security
>Affects Versions: 4.6, 5.5, 6.0, 6.1, 6.2, 6.3
>Reporter: Anshum Gupta
>Assignee: Anshum Gupta
>  Labels: commons-file-upload
> Attachments: SOLR-9819.patch
>
>
> We use Apache commons-fileupload 1.3.1. According to CVE-2016-3092 :
> "The MultipartStream class in Apache Commons Fileupload before 1.3.2, as used 
> in Apache Tomcat 7.x before 7.0.70, 8.x before 8.0.36, 8.5.x before 8.5.3, 
> and 9.x before 9.0.0.M7 and other products, allows remote attackers to cause 
> a denial of service (CPU consumption) via a long boundary string."
> [Source|http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-3092]
> We should upgrade to 1.3.2.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9819) Upgrade commons-fileupload to 1.3.2

2016-12-09 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-9819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736000#comment-15736000
 ] 

ASF subversion and git services commented on SOLR-9819:
---

Commit 416f2b7920b498f7b3ed07840e180c0d726f853b in lucene-solr's branch 
refs/heads/branch_5_5 from [~anshum]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=416f2b7 ]

SOLR-9819: Upgrade Apache commons-fileupload to 1.3.2, fixing a security 
vulnerability


> Upgrade commons-fileupload to 1.3.2
> ---
>
> Key: SOLR-9819
> URL: https://issues.apache.org/jira/browse/SOLR-9819
> Project: Solr
>  Issue Type: Improvement
>  Components: security
>Affects Versions: 4.6, 5.5, 6.0, 6.1, 6.2, 6.3
>Reporter: Anshum Gupta
>Assignee: Anshum Gupta
>  Labels: commons-file-upload
> Attachments: SOLR-9819.patch
>
>
> We use Apache commons-fileupload 1.3.1. According to CVE-2016-3092 :
> "The MultipartStream class in Apache Commons Fileupload before 1.3.2, as used 
> in Apache Tomcat 7.x before 7.0.70, 8.x before 8.0.36, 8.5.x before 8.5.3, 
> and 9.x before 9.0.0.M7 and other products, allows remote attackers to cause 
> a denial of service (CPU consumption) via a long boundary string."
> [Source|http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-3092]
> We should upgrade to 1.3.2.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9819) Upgrade commons-fileupload to 1.3.2

2016-12-09 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-9819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736001#comment-15736001
 ] 

ASF subversion and git services commented on SOLR-9819:
---

Commit b32f6904b3ef212d5cfd6c654338dd2d6af94a03 in lucene-solr's branch 
refs/heads/branch_5_5 from [~anshum]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b32f690 ]

SOLR-9819: Add new line to the end of SHA


> Upgrade commons-fileupload to 1.3.2
> ---
>
> Key: SOLR-9819
> URL: https://issues.apache.org/jira/browse/SOLR-9819
> Project: Solr
>  Issue Type: Improvement
>  Components: security
>Affects Versions: 4.6, 5.5, 6.0, 6.1, 6.2, 6.3
>Reporter: Anshum Gupta
>Assignee: Anshum Gupta
>  Labels: commons-file-upload
> Attachments: SOLR-9819.patch
>
>
> We use Apache commons-fileupload 1.3.1. According to CVE-2016-3092 :
> "The MultipartStream class in Apache Commons Fileupload before 1.3.2, as used 
> in Apache Tomcat 7.x before 7.0.70, 8.x before 8.0.36, 8.5.x before 8.5.3, 
> and 9.x before 9.0.0.M7 and other products, allows remote attackers to cause 
> a denial of service (CPU consumption) via a long boundary string."
> [Source|http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-3092]
> We should upgrade to 1.3.2.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6989) Implement MMapDirectory unmapping for coming Java 9 changes


[ 
https://issues.apache.org/jira/browse/LUCENE-6989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15735875#comment-15735875
 ] 

Uwe Schindler commented on LUCENE-6989:
---

I committed the fix for detection, but the build still fails a bit later. 
Groovy is also broken, so you cannot run tests.

> Implement MMapDirectory unmapping for coming Java 9 changes
> ---
>
> Key: LUCENE-6989
> URL: https://issues.apache.org/jira/browse/LUCENE-6989
> Project: Lucene - Core
>  Issue Type: Task
>  Components: core/store
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
>  Labels: Java9
> Fix For: 6.0
>
> Attachments: LUCENE-6989-disable5x.patch, 
> LUCENE-6989-disable5x.patch, LUCENE-6989-fixbuild148.patch, 
> LUCENE-6989-v2.patch, LUCENE-6989.patch, LUCENE-6989.patch, 
> LUCENE-6989.patch, LUCENE-6989.patch
>
>
> Originally, the sun.misc.Cleaner interface was declared as "critical API" in 
> [JEP 260|http://openjdk.java.net/jeps/260 ]
> Unfortunately the decission was changed in favor of a oficially supported 
> {{java.lang.ref.Cleaner}} API. A side effect of this change is to move all 
> existing {{sun.misc.Cleaner}} APIs into a non-exported package. This causes 
> our forceful unmapping to no longer work, because we can get the cleaner 
> instance via reflection, but trying to invoke it will throw one of the new 
> Jigsaw RuntimeException because it is completely inaccessible. This will make 
> our forceful unmapping fail. There are also no changes in Garbage collector, 
> the problem still exists.
> For more information see this [mailing list 
> thread|http://mail.openjdk.java.net/pipermail/core-libs-dev/2016-January/thread.html#38243].
> This commit will likely be done, making our unmapping efforts no longer 
> working. Alan Bateman is aware of this issue and will open a new issue at 
> OpenJDK to allow forceful unmapping without using the now private 
> sun.misc.Cleaner. The idea is to let the internal class sun.misc.Cleaner 
> implement the Runable interface, so we can simply cast to runable and call 
> the run() method to unmap. The code would then work. This will lead to minor 
> changes in our unmapper in MMapDirectory: An instanceof check and casting if 
> possible.
> I opened this issue to keep track and implement the changes as soon as 
> possible, so people will have working unmapping when java 9 comes out. 
> Current Lucene versions will no longer work with Java 9.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Linux (64bit/jdk1.8.0_112) - Build # 18483 - Unstable!

2016-12-09 Thread ASF subversion and git services (JIRA)

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/18483/
Java: 64bit/jdk1.8.0_112 -XX:+UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.cloud.ShardSplitTest.testSplitAfterFailedSplit

Error Message:
expected:<1> but was:<2>

Stack Trace:
java.lang.AssertionError: expected:<1> but was:<2>
at 
__randomizedtesting.SeedInfo.seed([229A4E75B001EFFF:DBD7DDDA8C74A275]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at 
org.apache.solr.cloud.ShardSplitTest.testSplitAfterFailedSplit(ShardSplitTest.java:279)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at

[jira] [Commented] (LUCENE-6989) Implement MMapDirectory unmapping for coming Java 9 changes


[ 
https://issues.apache.org/jira/browse/LUCENE-6989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15735868#comment-15735868
 ] 

ASF subversion and git services commented on LUCENE-6989:
-

Commit d00ab65dccbef7add8b433a338c2ec3cea980916 in lucene-solr's branch 
refs/heads/branch_6x from [~thetaphi]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=d00ab65 ]

LUCENE-6989: Fix Exception handling in MMapDirectory's unmap hack support code 
to work with Java 9's new InaccessibleObjectException that does not extend 
ReflectiveAccessException in Java 9.


> Implement MMapDirectory unmapping for coming Java 9 changes
> ---
>
> Key: LUCENE-6989
> URL: https://issues.apache.org/jira/browse/LUCENE-6989
> Project: Lucene - Core
>  Issue Type: Task
>  Components: core/store
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
>  Labels: Java9
> Fix For: 6.0
>
> Attachments: LUCENE-6989-disable5x.patch, 
> LUCENE-6989-disable5x.patch, LUCENE-6989-fixbuild148.patch, 
> LUCENE-6989-v2.patch, LUCENE-6989.patch, LUCENE-6989.patch, 
> LUCENE-6989.patch, LUCENE-6989.patch
>
>
> Originally, the sun.misc.Cleaner interface was declared as "critical API" in 
> [JEP 260|http://openjdk.java.net/jeps/260 ]
> Unfortunately the decission was changed in favor of a oficially supported 
> {{java.lang.ref.Cleaner}} API. A side effect of this change is to move all 
> existing {{sun.misc.Cleaner}} APIs into a non-exported package. This causes 
> our forceful unmapping to no longer work, because we can get the cleaner 
> instance via reflection, but trying to invoke it will throw one of the new 
> Jigsaw RuntimeException because it is completely inaccessible. This will make 
> our forceful unmapping fail. There are also no changes in Garbage collector, 
> the problem still exists.
> For more information see this [mailing list 
> thread|http://mail.openjdk.java.net/pipermail/core-libs-dev/2016-January/thread.html#38243].
> This commit will likely be done, making our unmapping efforts no longer 
> working. Alan Bateman is aware of this issue and will open a new issue at 
> OpenJDK to allow forceful unmapping without using the now private 
> sun.misc.Cleaner. The idea is to let the internal class sun.misc.Cleaner 
> implement the Runable interface, so we can simply cast to runable and call 
> the run() method to unmap. The code would then work. This will lead to minor 
> changes in our unmapper in MMapDirectory: An instanceof check and casting if 
> possible.
> I opened this issue to keep track and implement the changes as soon as 
> possible, so people will have working unmapping when java 9 comes out. 
> Current Lucene versions will no longer work with Java 9.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6989) Implement MMapDirectory unmapping for coming Java 9 changes

2016-12-09 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-6989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15735865#comment-15735865
 ] 

ASF subversion and git services commented on LUCENE-6989:
-

Commit 22d04a7c1149c1af42dc2890a416fc45e4d0aa5e in lucene-solr's branch 
refs/heads/master from [~thetaphi]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=22d04a7 ]

LUCENE-6989: Fix Exception handling in MMapDirectory's unmap hack support code 
to work with Java 9's new InaccessibleObjectException that does not extend 
ReflectiveAccessException in Java 9.


> Implement MMapDirectory unmapping for coming Java 9 changes
> ---
>
> Key: LUCENE-6989
> URL: https://issues.apache.org/jira/browse/LUCENE-6989
> Project: Lucene - Core
>  Issue Type: Task
>  Components: core/store
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
>  Labels: Java9
> Fix For: 6.0
>
> Attachments: LUCENE-6989-disable5x.patch, 
> LUCENE-6989-disable5x.patch, LUCENE-6989-fixbuild148.patch, 
> LUCENE-6989-v2.patch, LUCENE-6989.patch, LUCENE-6989.patch, 
> LUCENE-6989.patch, LUCENE-6989.patch
>
>
> Originally, the sun.misc.Cleaner interface was declared as "critical API" in 
> [JEP 260|http://openjdk.java.net/jeps/260 ]
> Unfortunately the decission was changed in favor of a oficially supported 
> {{java.lang.ref.Cleaner}} API. A side effect of this change is to move all 
> existing {{sun.misc.Cleaner}} APIs into a non-exported package. This causes 
> our forceful unmapping to no longer work, because we can get the cleaner 
> instance via reflection, but trying to invoke it will throw one of the new 
> Jigsaw RuntimeException because it is completely inaccessible. This will make 
> our forceful unmapping fail. There are also no changes in Garbage collector, 
> the problem still exists.
> For more information see this [mailing list 
> thread|http://mail.openjdk.java.net/pipermail/core-libs-dev/2016-January/thread.html#38243].
> This commit will likely be done, making our unmapping efforts no longer 
> working. Alan Bateman is aware of this issue and will open a new issue at 
> OpenJDK to allow forceful unmapping without using the now private 
> sun.misc.Cleaner. The idea is to let the internal class sun.misc.Cleaner 
> implement the Runable interface, so we can simply cast to runable and call 
> the run() method to unmap. The code would then work. This will lead to minor 
> changes in our unmapper in MMapDirectory: An instanceof check and casting if 
> possible.
> I opened this issue to keep track and implement the changes as soon as 
> possible, so people will have working unmapping when java 9 comes out. 
> Current Lucene versions will no longer work with Java 9.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6989) Implement MMapDirectory unmapping for coming Java 9 changes


[ 
https://issues.apache.org/jira/browse/LUCENE-6989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15735844#comment-15735844
 ] 

Uwe Schindler commented on LUCENE-6989:
---

With this patch the class loads again, although it does not work:

{noformat}
   [junit4] Suite: org.apache.lucene.store.TestMultiMMap
   [junit4] IGNOR/A 0.03s | TestMultiMMap.testDeleteFile
   [junit4]> Assumption #1: test requires a jre that supports unmapping: 
Unmapping is not supported on this platform, because internal Java APIs are not 
compatible to this Lucene version: 
java.lang.reflect.InaccessibleObjectException: Unable to make public 
jdk.internal.ref.Cleaner java.nio.DirectByteBuffer.cleaner() accessible: module 
java.base does not "opens java.nio" to unnamed module @4823fcdd
{noformat}

> Implement MMapDirectory unmapping for coming Java 9 changes
> ---
>
> Key: LUCENE-6989
> URL: https://issues.apache.org/jira/browse/LUCENE-6989
> Project: Lucene - Core
>  Issue Type: Task
>  Components: core/store
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
>  Labels: Java9
> Fix For: 6.0
>
> Attachments: LUCENE-6989-disable5x.patch, 
> LUCENE-6989-disable5x.patch, LUCENE-6989-fixbuild148.patch, 
> LUCENE-6989-v2.patch, LUCENE-6989.patch, LUCENE-6989.patch, 
> LUCENE-6989.patch, LUCENE-6989.patch
>
>
> Originally, the sun.misc.Cleaner interface was declared as "critical API" in 
> [JEP 260|http://openjdk.java.net/jeps/260 ]
> Unfortunately the decission was changed in favor of a oficially supported 
> {{java.lang.ref.Cleaner}} API. A side effect of this change is to move all 
> existing {{sun.misc.Cleaner}} APIs into a non-exported package. This causes 
> our forceful unmapping to no longer work, because we can get the cleaner 
> instance via reflection, but trying to invoke it will throw one of the new 
> Jigsaw RuntimeException because it is completely inaccessible. This will make 
> our forceful unmapping fail. There are also no changes in Garbage collector, 
> the problem still exists.
> For more information see this [mailing list 
> thread|http://mail.openjdk.java.net/pipermail/core-libs-dev/2016-January/thread.html#38243].
> This commit will likely be done, making our unmapping efforts no longer 
> working. Alan Bateman is aware of this issue and will open a new issue at 
> OpenJDK to allow forceful unmapping without using the now private 
> sun.misc.Cleaner. The idea is to let the internal class sun.misc.Cleaner 
> implement the Runable interface, so we can simply cast to runable and call 
> the run() method to unmap. The code would then work. This will lead to minor 
> changes in our unmapper in MMapDirectory: An instanceof check and casting if 
> possible.
> I opened this issue to keep track and implement the changes as soon as 
> possible, so people will have working unmapping when java 9 comes out. 
> Current Lucene versions will no longer work with Java 9.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-6989) Implement MMapDirectory unmapping for coming Java 9 changes


 [ 
https://issues.apache.org/jira/browse/LUCENE-6989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-6989:
--
Attachment: LUCENE-6989-fixbuild148.patch

Here is a patch for the Lucene issue that prevents loading of class on the 
crazy new InaccessibleObjectException (extends RuntimeException). We should 
apply this in any case to all open branches.

This does not fix the issue in build 148 of Java 9, which makes unmapping 
impossible. I am working on resolving this with OpenJDK/Oracle.

> Implement MMapDirectory unmapping for coming Java 9 changes
> ---
>
> Key: LUCENE-6989
> URL: https://issues.apache.org/jira/browse/LUCENE-6989
> Project: Lucene - Core
>  Issue Type: Task
>  Components: core/store
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
>  Labels: Java9
> Fix For: 6.0
>
> Attachments: LUCENE-6989-disable5x.patch, 
> LUCENE-6989-disable5x.patch, LUCENE-6989-fixbuild148.patch, 
> LUCENE-6989-v2.patch, LUCENE-6989.patch, LUCENE-6989.patch, 
> LUCENE-6989.patch, LUCENE-6989.patch
>
>
> Originally, the sun.misc.Cleaner interface was declared as "critical API" in 
> [JEP 260|http://openjdk.java.net/jeps/260 ]
> Unfortunately the decission was changed in favor of a oficially supported 
> {{java.lang.ref.Cleaner}} API. A side effect of this change is to move all 
> existing {{sun.misc.Cleaner}} APIs into a non-exported package. This causes 
> our forceful unmapping to no longer work, because we can get the cleaner 
> instance via reflection, but trying to invoke it will throw one of the new 
> Jigsaw RuntimeException because it is completely inaccessible. This will make 
> our forceful unmapping fail. There are also no changes in Garbage collector, 
> the problem still exists.
> For more information see this [mailing list 
> thread|http://mail.openjdk.java.net/pipermail/core-libs-dev/2016-January/thread.html#38243].
> This commit will likely be done, making our unmapping efforts no longer 
> working. Alan Bateman is aware of this issue and will open a new issue at 
> OpenJDK to allow forceful unmapping without using the now private 
> sun.misc.Cleaner. The idea is to let the internal class sun.misc.Cleaner 
> implement the Runable interface, so we can simply cast to runable and call 
> the run() method to unmap. The code would then work. This will lead to minor 
> changes in our unmapper in MMapDirectory: An instanceof check and casting if 
> possible.
> I opened this issue to keep track and implement the changes as soon as 
> possible, so people will have working unmapping when java 9 comes out. 
> Current Lucene versions will no longer work with Java 9.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6989) Implement MMapDirectory unmapping for coming Java 9 changes


[ 
https://issues.apache.org/jira/browse/LUCENE-6989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15735817#comment-15735817
 ] 

Uwe Schindler commented on LUCENE-6989:
---

In Java 9 build 148, the hack broke again, but unfortunately fatal. There are 2 
problems:
- Java 9 now prevents access to the ByteBuffer. This is the total desaster
- Lucene has a bug that it fails when testing the availability of unmapping, 
because setAccessible throws InaccessibleObjectException that is subclass of 
RuntimeException and not ReflectiveOperationException. We have to fix this, as 
otherwise loading of the class MMapDircetory fails completely.

> Implement MMapDirectory unmapping for coming Java 9 changes
> ---
>
> Key: LUCENE-6989
> URL: https://issues.apache.org/jira/browse/LUCENE-6989
> Project: Lucene - Core
>  Issue Type: Task
>  Components: core/store
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
>  Labels: Java9
> Fix For: 6.0
>
> Attachments: LUCENE-6989-disable5x.patch, 
> LUCENE-6989-disable5x.patch, LUCENE-6989-v2.patch, LUCENE-6989.patch, 
> LUCENE-6989.patch, LUCENE-6989.patch, LUCENE-6989.patch
>
>
> Originally, the sun.misc.Cleaner interface was declared as "critical API" in 
> [JEP 260|http://openjdk.java.net/jeps/260 ]
> Unfortunately the decission was changed in favor of a oficially supported 
> {{java.lang.ref.Cleaner}} API. A side effect of this change is to move all 
> existing {{sun.misc.Cleaner}} APIs into a non-exported package. This causes 
> our forceful unmapping to no longer work, because we can get the cleaner 
> instance via reflection, but trying to invoke it will throw one of the new 
> Jigsaw RuntimeException because it is completely inaccessible. This will make 
> our forceful unmapping fail. There are also no changes in Garbage collector, 
> the problem still exists.
> For more information see this [mailing list 
> thread|http://mail.openjdk.java.net/pipermail/core-libs-dev/2016-January/thread.html#38243].
> This commit will likely be done, making our unmapping efforts no longer 
> working. Alan Bateman is aware of this issue and will open a new issue at 
> OpenJDK to allow forceful unmapping without using the now private 
> sun.misc.Cleaner. The idea is to let the internal class sun.misc.Cleaner 
> implement the Runable interface, so we can simply cast to runable and call 
> the run() method to unmap. The code would then work. This will lead to minor 
> changes in our unmapper in MMapDirectory: An instanceof check and casting if 
> possible.
> I opened this issue to keep track and implement the changes as soon as 
> possible, so people will have working unmapping when java 9 comes out. 
> Current Lucene versions will no longer work with Java 9.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

RE: JDK 9 b148 including a refresh of the module system is available on java.net

Hi,

 

Sorry my message was incorrect:

There are 2 bugs in Lucene and Java:

-  In Lucene the unmap hack’s exception handling falls over the stupid 
InaccessibleObjectException does not extend ReflectiveOperationException issue 
(I complained about that several times). I can fix this, but it’s horrible.

-  Java now hides internal of ByteBuffer completely, although the JEP 
about Unsafe & Co. said which Java APIs are excluded from the lock down. Next 
to Unsafe, also ByteBuffer was on the list of “still allowed APIs where the 
Java 9 backwards layer helps”. MappedByteBuffer unmapping is one of the most 
important parts in Lucene that must be supported, otherwise it is impossible to 
use Lucene, Solr, or Elasticsearch with Java 9 in a performant way. We can also 
not require from our users to add strange command line options to their JVMs.

 

Uwe

 

-

Uwe Schindler

Achterdiek 19, D-28357 Bremen

  http://www.thetaphi.de

eMail: u...@thetaphi.de

 

From: Uwe Schindler [mailto:u...@thetaphi.de] 
Sent: Friday, December 9, 2016 5:53 PM
To: dev@lucene.apache.org; 'Rory O'Donnell' 
Cc: 'Dalibor Topic' ; balchandra.vai...@oracle.com; 
abdul.kolarku...@oracle.com; 'Dawid Weiss' 
Subject: RE: JDK 9 b148 including a refresh of the module system is available 
on java.net

 

Hi,

 

build 148 broke the unmap hack. The problem is an additional bug in Java: The 
unmap hack should be more unspecific on the Exception Type 
(InaccessibleObjectException does not extend ReflectiveOperationException). I 
will change this in an issue, but the general problem persists:

 

With the buil 148 version we can no longer unmap byte buffers:

 

java.lang.reflect.InaccessibleObjectException: Unable to make public 
jdk.internal.ref.Cleaner java.nio.DirectByteBuffer.cleaner() accessible: module 
java.base does not "opens java.nio" to unnamed module @10b73cf

   [junit4]>   at 
java.base/jdk.internal.reflect.Reflection.throwInaccessibleObjectException(Reflection.java:427)

   [junit4]>   at 
java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:201)

   [junit4]>   at 
java.base/java.lang.reflect.Method.checkCanSetAccessible(Method.java:192)

   [junit4]>   at 
java.base/java.lang.reflect.Method.setAccessible(Method.java:186)

   [junit4]>   at 
org.apache.lucene.store.MMapDirectory.unmapHackImpl(MMapDirectory.java:345)

   [junit4]>   at 
java.base/java.security.AccessController.doPrivileged(Native Method)

   [junit4]>   at 
org.apache.lucene.store.MMapDirectory.(MMapDirectory.java:326)

   [junit4]>   ... 42 moreThrowable #2: java.lang.NullPointerException

   [junit4]>   at 
org.apache.lucene.search.TestDateSort.tearDown(TestDateSort.java:71)

   [junit4]>   at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

   [junit4]>   at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

   [junit4]>   at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

   [junit4]>   at 
java.base/java.lang.reflect.Method.invoke(Method.java:538)

   [junit4]>   at java.base/java.lang.Thread.run(Thread.java:844)

 

 

-

Uwe Schindler

Achterdiek 19, D-28357 Bremen

http://www.thetaphi.de  

eMail:   u...@thetaphi.de

 

From: Uwe Schindler [  mailto:u...@thetaphi.de] 
Sent: Friday, December 9, 2016 4:35 PM
To: 'Rory O'Donnell' <  
rory.odonn...@oracle.com>
Cc: 'Dalibor Topic' <  
dalibor.to...@oracle.com>;   
balchandra.vai...@oracle.com;   
abdul.kolarku...@oracle.com; 'Dawid Weiss' < 
 dawid.we...@cs.put.poznan.pl>;  
 dev@lucene.apache.org
Subject: RE: JDK 9 b148 including a refresh of the module system is available 
on java.net

 

Hi Rory,

 

thanks for the message! I put build 148 in the Jenkins test rotation. Let’s see 
what happens.

 

Uwe

 

-

Uwe Schindler

Achterdiek 19, D-28357 Bremen

http://www.thetaphi.de  

eMail:   u...@thetaphi.de

 

From: Rory O'Donnell [  
mailto:rory.odonn...@oracle.com] 
Sent: Friday, December 9, 2016 11:34 AM
To:   u...@thetaphi.de
Cc:   rory.odonn...@oracle.com; Dalibor Topic 
<  dalibor.to...@oracle.com>;  
 balchandra.vai...@oracle.com;  
 abdul.kolarku...@oracle.com; Dawid

[jira] [Updated] (LUCENE-7589) Prevent outliers from raising the number of bits of everyone with numeric doc values

2016-12-09 Thread Adrien Grand (JIRA)


 [ 
https://issues.apache.org/jira/browse/LUCENE-7589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adrien Grand updated LUCENE-7589:
-
Attachment: LUCENE-7589.patch

Here is a patch. The doc values consumer computes space usage both for the case 
that all values use the same number of bits per value and for the case that 
values are split into blocks of 16384 values. And if using blocks proves to 
save 10% disk usage or more, then it encodes blocks with their own required 
number of bits per value.

I kept a rather high value of the block size, since this impl can only jump 
forward {{blockSize}} documents at a time, so a high value like 16384 hopefully 
keeps performance good, but in the future we might want to look into leveraging 
the sequential access pattern even more (to do run-length encoding for 
instance) and maybe have eg. a skip list to handle the big jumps, like postings 
do. I think that patch is a good first (baby) step towards that direction.

> Prevent outliers from raising the number of bits of everyone with numeric doc 
> values
> 
>
> Key: LUCENE-7589
> URL: https://issues.apache.org/jira/browse/LUCENE-7589
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Adrien Grand
>Assignee: Adrien Grand
>Priority: Minor
> Attachments: LUCENE-7589.patch
>
>
> Today we encode entire segments with a single number of bits per value. It 
> was done this way because it was faster, but it also means a single outlier 
> can significantly increase the space requirements. I think we should have 
> protection against that.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

RE: JDK 9 b148 including a refresh of the module system is available on java.net

Hi,

 

build 148 broke the unmap hack. The problem is an additional bug in Java: The 
unmap hack should be more unspecific on the Exception Type 
(InaccessibleObjectException does not extend ReflectiveOperationException). I 
will change this in an issue, but the general problem persists:

 

With the buil 148 version we can no longer unmap byte buffers:

 

java.lang.reflect.InaccessibleObjectException: Unable to make public 
jdk.internal.ref.Cleaner java.nio.DirectByteBuffer.cleaner() accessible: module 
java.base does not "opens java.nio" to unnamed module @10b73cf

   [junit4]>   at 
java.base/jdk.internal.reflect.Reflection.throwInaccessibleObjectException(Reflection.java:427)

   [junit4]>   at 
java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:201)

   [junit4]>   at 
java.base/java.lang.reflect.Method.checkCanSetAccessible(Method.java:192)

   [junit4]>   at 
java.base/java.lang.reflect.Method.setAccessible(Method.java:186)

   [junit4]>   at 
org.apache.lucene.store.MMapDirectory.unmapHackImpl(MMapDirectory.java:345)

   [junit4]>   at 
java.base/java.security.AccessController.doPrivileged(Native Method)

   [junit4]>   at 
org.apache.lucene.store.MMapDirectory.(MMapDirectory.java:326)

   [junit4]>   ... 42 moreThrowable #2: java.lang.NullPointerException

   [junit4]>   at 
org.apache.lucene.search.TestDateSort.tearDown(TestDateSort.java:71)

   [junit4]>   at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

   [junit4]>   at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

   [junit4]>   at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

   [junit4]>   at 
java.base/java.lang.reflect.Method.invoke(Method.java:538)

   [junit4]>   at java.base/java.lang.Thread.run(Thread.java:844)

 

 

-

Uwe Schindler

Achterdiek 19, D-28357 Bremen

http://www.thetaphi.de  

eMail: u...@thetaphi.de

 

From: Uwe Schindler [mailto:u...@thetaphi.de] 
Sent: Friday, December 9, 2016 4:35 PM
To: 'Rory O'Donnell' 
Cc: 'Dalibor Topic' ; balchandra.vai...@oracle.com; 
abdul.kolarku...@oracle.com; 'Dawid Weiss' ; 
dev@lucene.apache.org
Subject: RE: JDK 9 b148 including a refresh of the module system is available 
on java.net

 

Hi Rory,

 

thanks for the message! I put build 148 in the Jenkins test rotation. Let’s see 
what happens.

 

Uwe

 

-

Uwe Schindler

Achterdiek 19, D-28357 Bremen

http://www.thetaphi.de  

eMail:   u...@thetaphi.de

 

From: Rory O'Donnell [  
mailto:rory.odonn...@oracle.com] 
Sent: Friday, December 9, 2016 11:34 AM
To:   u...@thetaphi.de
Cc:   rory.odonn...@oracle.com; Dalibor Topic 
<  dalibor.to...@oracle.com>;  
 balchandra.vai...@oracle.com;  
 abdul.kolarku...@oracle.com; Dawid Weiss < 
 dawid.we...@cs.put.poznan.pl>;  
 dev@lucene.apache.org
Subject: JDK 9 b148 including a refresh of the module system is available on 
java.net

 

 

Hi Uwe & Dawid, 


JDK 9 build b148   includes an important 
Refresh of the module system [1] , summary of  changes are listed here  
 . 

This refresh includes a disruptive change that is important to understand. 

For those that have been trying out modules with regular JDK 9 builds then be 
aware that `requires public` changes to `requires transitive`. In addition, the 
binary representation of the module declaration (module-info.class) has changed 
so that you need to recompile any modules that were compiled with previous JDK 
9 builds. 

As things stand today in JDK 9 then you use setAccessible to break into 
non-public elements of any type in exported packages. However, it cannot be 
used to break into any type in non-exported package. The current specified 
behavior was a compromise for the initial integration of the module system. It 
is of course not very satisfactory, hence the #AwkwardStrongEncapsulation issue 
[2] on the JSR 376 issues list. With the updated proposal in the JSR, this 
refresh changes setAccessible further so that it cannot be used to break into 
non-public types, or non-public elements of public types, in exported packages. 
Code that uses setAccessible to hack into the private constructor of 
java.lang.invoke.MethodHandles.Lookup will be disappointed for example.

This change will expose hacks in many existing libraries and

[jira] [Commented] (SOLR-5944) Support updates of numeric DocValues


[ 
https://issues.apache.org/jira/browse/SOLR-5944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15735756#comment-15735756
 ] 

Hoss Man commented on SOLR-5944:


bq. So, therefore, I think we should be checking if the field exists or not, 
irrespective of explicit or dynamic. And if the field doesn't exist, we should 
just delegate to a regular atomic update. Do you think it makes sense?

IIUC, in a nutshell: all of my questions about why we're treating dynamicFields 
as special will no longer be relevent, because dynamicFields will no longer be 
considered special, because they were never really special to begin with, they 
just seemed that way because the only testing of non-dynamic fields 
assumed/required them to have a schema default.  we'll remove that assumption 
from both the tests and code, and treat all fields the same.

If my understanding is correct, then yes you're plan to move forward seems 
sound.


> Support updates of numeric DocValues
> 
>
> Key: SOLR-5944
> URL: https://issues.apache.org/jira/browse/SOLR-5944
> Project: Solr
>  Issue Type: New Feature
>Reporter: Ishan Chattopadhyaya
>Assignee: Shalin Shekhar Mangar
> Attachments: DUP.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> TestStressInPlaceUpdates.eb044ac71.beast-167-failure.stdout.txt, 
> TestStressInPlaceUpdates.eb044ac71.beast-587-failure.stdout.txt, 
> TestStressInPlaceUpdates.eb044ac71.failures.tar.gz, defensive-checks.log.gz, 
> demo-why-dynamic-fields-cannot-be-inplace-updated-first-time.patch, 
> hoss.62D328FA1DEA57FD.fail.txt, hoss.62D328FA1DEA57FD.fail2.txt, 
> hoss.62D328FA1DEA57FD.fail3.txt, hoss.D768DD9443A98DC.fail.txt, 
> hoss.D768DD9443A98DC.pass.txt
>
>
> LUCENE-5189 introduced support for updates to numeric docvalues. It would be 
> really nice to have Solr support this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7588) A parallel DrillSideways implementation


 [ 
https://issues.apache.org/jira/browse/LUCENE-7588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Emmanuel Keller updated LUCENE-7588:

Attachment: LUCENE-7588.patch

> A parallel DrillSideways implementation
> ---
>
> Key: LUCENE-7588
> URL: https://issues.apache.org/jira/browse/LUCENE-7588
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: master (7.0), 6.3.1
>Reporter: Emmanuel Keller
>Priority: Minor
>  Labels: facet, faceting
> Fix For: master (7.0), 6.3.1
>
> Attachments: LUCENE-7588.patch
>
>
> Currently DrillSideways implementation is based on the single threaded 
> IndexSearcher.search(Query query, Collector results).
> On large document set, the single threaded collection can be really slow.
> The ParallelDrillSideways implementation could:
> 1. Use the CollectionManager based method IndexSearcher.search(Query query, 
> CollectorManager collectorManager)  to get the benefits of multithreading on 
> index segments,
> 2. Compute each DrillSideway subquery on a single thread.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-7589) Prevent outliers from raising the number of bits of everyone with numeric doc values

2016-12-09 Thread Adrien Grand (JIRA)

Adrien Grand created LUCENE-7589:


 Summary: Prevent outliers from raising the number of bits of 
everyone with numeric doc values
 Key: LUCENE-7589
 URL: https://issues.apache.org/jira/browse/LUCENE-7589
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Adrien Grand
Assignee: Adrien Grand
Priority: Minor


Today we encode entire segments with a single number of bits per value. It was 
done this way because it was faster, but it also means a single outlier can 
significantly increase the space requirements. I think we should have 
protection against that.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9132) Cut over AbstractDistribZkTestCase tests to SolrCloudTestCase

2016-12-09 Thread Alan Woodward (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-9132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Woodward updated SOLR-9132:

Attachment: SOLR-9132-overseertests.patch

Latest patch, cutting over some more tests.  This also fixes SOLR-9767.

> Cut over AbstractDistribZkTestCase tests to SolrCloudTestCase
> -
>
> Key: SOLR-9132
> URL: https://issues.apache.org/jira/browse/SOLR-9132
> Project: Solr
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Attachments: SOLR-9132-deletereplicas.patch, 
> SOLR-9132-overseertests.patch, SOLR-9132-recovery.patch, 
> SOLR-9132-rules.patch, SOLR-9132.patch, 
> TEST-org.apache.solr.cloud.TestDeleteCollectionOnDownNodes.xml
>
>
> We need to remove AbstractDistribZkTestCase if we want to move away from 
> legacy cloud configurations.  This issue is for migrating tests to 
> SolrCloudTestCase instead.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Windows (64bit/jdk1.8.0_112) - Build # 6276 - Still Unstable!

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/6276/
Java: 64bit/jdk1.8.0_112 -XX:-UseCompressedOops -XX:+UseSerialGC

1 tests failed.
FAILED:  
org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.testDelegationTokenCancelFail

Error Message:
expected:<200> but was:<404>

Stack Trace:
java.lang.AssertionError: expected:<200> but was:<404>
at 
__randomizedtesting.SeedInfo.seed([F3E0AA87C55439D3:9B5F9FAD15CE2B3F]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at 
org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.cancelDelegationToken(TestSolrCloudWithDelegationTokens.java:140)
at 
org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.testDelegationTokenCancelFail(TestSolrCloudWithDelegationTokens.java:294)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at

[jira] [Updated] (LUCENE-7588) A parallel DrillSideways implementation


 [ 
https://issues.apache.org/jira/browse/LUCENE-7588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Emmanuel Keller updated LUCENE-7588:

Lucene Fields: New,Patch Available  (was: New)

> A parallel DrillSideways implementation
> ---
>
> Key: LUCENE-7588
> URL: https://issues.apache.org/jira/browse/LUCENE-7588
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: master (7.0), 6.3.1
>Reporter: Emmanuel Keller
>Priority: Minor
>  Labels: facet, faceting
> Fix For: master (7.0), 6.3.1
>
>
> Currently DrillSideways implementation is based on the single threaded 
> IndexSearcher.search(Query query, Collector results).
> On large document set, the single threaded collection can be really slow.
> The ParallelDrillSideways implementation could:
> 1. Use the CollectionManager based method IndexSearcher.search(Query query, 
> CollectorManager collectorManager)  to get the benefits of multithreading on 
> index segments,
> 2. Compute each DrillSideway subquery on a single thread.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7588) A parallel DrillSideways implementation


 [ 
https://issues.apache.org/jira/browse/LUCENE-7588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Emmanuel Keller updated LUCENE-7588:

Labels: facet faceting  (was: )

> A parallel DrillSideways implementation
> ---
>
> Key: LUCENE-7588
> URL: https://issues.apache.org/jira/browse/LUCENE-7588
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: master (7.0), 6.3.1
>Reporter: Emmanuel Keller
>Priority: Minor
>  Labels: facet, faceting
> Fix For: master (7.0), 6.3.1
>
>
> Currently DrillSideways implementation is based on the single threaded 
> IndexSearcher.search(Query query, Collector results).
> On large document set, the single threaded collection can be really slow.
> The ParallelDrillSideways implementation could:
> 1. Use the CollectionManager based method IndexSearcher.search(Query query, 
> CollectorManager collectorManager)  to get the benefits of multithreading on 
> index segments,
> 2. Compute each DrillSideway subquery on a single thread.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-7588) A parallel DrillSideways implementation

2016-12-09 Thread ASF subversion and git services (JIRA)

Emmanuel Keller created LUCENE-7588:
---

 Summary: A parallel DrillSideways implementation
 Key: LUCENE-7588
 URL: https://issues.apache.org/jira/browse/LUCENE-7588
 Project: Lucene - Core
  Issue Type: Improvement
Affects Versions: master (7.0), 6.3.1
Reporter: Emmanuel Keller
Priority: Minor
 Fix For: master (7.0), 6.3.1


Currently DrillSideways implementation is based on the single threaded 
IndexSearcher.search(Query query, Collector results).

On large document set, the single threaded collection can be really slow.

The ParallelDrillSideways implementation could:

1. Use the CollectionManager based method IndexSearcher.search(Query query, 
CollectorManager collectorManager)  to get the benefits of multithreading on 
index segments,
2. Compute each DrillSideway subquery on a single thread.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8029) Modernize and standardize Solr APIs


[ 
https://issues.apache.org/jira/browse/SOLR-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15735664#comment-15735664
 ] 

ASF subversion and git services commented on SOLR-8029:
---

Commit cedc63d46e5aca384b8b34fbe7c51568ffbe671a in lucene-solr's branch 
refs/heads/apiv2 from [~ctargett]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=cedc63d ]

SOLR-8029: Modify schemaEdit specs; add include file for analyzers


> Modernize and standardize Solr APIs
> ---
>
> Key: SOLR-8029
> URL: https://issues.apache.org/jira/browse/SOLR-8029
> Project: Solr
>  Issue Type: Improvement
>Affects Versions: 6.0
>Reporter: Noble Paul
>Assignee: Noble Paul
>  Labels: API, EaseOfUse
> Fix For: 6.0
>
> Attachments: SOLR-8029.patch, SOLR-8029.patch, SOLR-8029.patch, 
> SOLR-8029.patch
>
>
> Solr APIs have organically evolved and they are sometimes inconsistent with 
> each other or not in sync with the widely followed conventions of HTTP 
> protocol. Trying to make incremental changes to make them modern is like 
> applying band-aid. So, we have done a complete rethink of what the APIs 
> should be. The most notable aspects of the API are as follows:
> The new set of APIs will be placed under a new path {{/solr2}}. The legacy 
> APIs will continue to work under the {{/solr}} path as they used to and they 
> will be eventually deprecated.
> There are 4 types of requests in the new API 
> * {{/v2//*}} : Hit a collection directly or manage 
> collections/shards/replicas 
> * {{/v2//*}} : Hit a core directly or manage cores 
> * {{/v2/cluster/*}} : Operations on cluster not pertaining to any collection 
> or core. e.g: security, overseer ops etc
> This will be released as part of a major release. Check the link given below 
> for the full specification.  Your comments are welcome
> [Solr API version 2 Specification | http://bit.ly/1JYsBMQ]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8029) Modernize and standardize Solr APIs

2016-12-09 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15735663#comment-15735663
 ] 

ASF subversion and git services commented on SOLR-8029:
---

Commit bf89a7a405138b39d91d4dc31a9a920e4ecf6ff3 in lucene-solr's branch 
refs/heads/apiv2 from [~ctargett]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=bf89a7a ]

SOLR-8029: Update 'cluster' specs with descriptions and links to docs


> Modernize and standardize Solr APIs
> ---
>
> Key: SOLR-8029
> URL: https://issues.apache.org/jira/browse/SOLR-8029
> Project: Solr
>  Issue Type: Improvement
>Affects Versions: 6.0
>Reporter: Noble Paul
>Assignee: Noble Paul
>  Labels: API, EaseOfUse
> Fix For: 6.0
>
> Attachments: SOLR-8029.patch, SOLR-8029.patch, SOLR-8029.patch, 
> SOLR-8029.patch
>
>
> Solr APIs have organically evolved and they are sometimes inconsistent with 
> each other or not in sync with the widely followed conventions of HTTP 
> protocol. Trying to make incremental changes to make them modern is like 
> applying band-aid. So, we have done a complete rethink of what the APIs 
> should be. The most notable aspects of the API are as follows:
> The new set of APIs will be placed under a new path {{/solr2}}. The legacy 
> APIs will continue to work under the {{/solr}} path as they used to and they 
> will be eventually deprecated.
> There are 4 types of requests in the new API 
> * {{/v2//*}} : Hit a collection directly or manage 
> collections/shards/replicas 
> * {{/v2//*}} : Hit a core directly or manage cores 
> * {{/v2/cluster/*}} : Operations on cluster not pertaining to any collection 
> or core. e.g: security, overseer ops etc
> This will be released as part of a major release. Check the link given below 
> for the full specification.  Your comments are welcome
> [Solr API version 2 Specification | http://bit.ly/1JYsBMQ]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-EA] Lucene-Solr-6.x-Linux (32bit/jdk-9-ea+148) - Build # 2379 - Failure!

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/2379/
Java: 32bit/jdk-9-ea+148 -client -XX:+UseConcMarkSweepGC

959 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.lucene.index.TestCodecs

Error Message:
Clean up static fields (in @AfterClass?) and null them, your test still has 
references to classes of which the sizes cannot be measured due to security 
restrictions or Java 9 module encapsulation:   - private static 
java.lang.String[] org.apache.lucene.index.TestCodecs.fieldNames

Stack Trace:
junit.framework.AssertionFailedError: Clean up static fields (in @AfterClass?) 
and null them, your test still has references to classes of which the sizes 
cannot be measured due to security restrictions or Java 9 module encapsulation:
  - private static java.lang.String[] 
org.apache.lucene.index.TestCodecs.fieldNames
at __randomizedtesting.SeedInfo.seed([69F8ACEB4E10CEB5]:0)
at 
com.carrotsearch.randomizedtesting.rules.StaticFieldsInvariantRule$1.afterAlways(StaticFieldsInvariantRule.java:146)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.base/java.lang.Thread.run(Thread.java:844)
Caused by: java.lang.IllegalStateException: Unable to access 'private final 
byte[] java.lang.String.value' to estimate memory usage
at 
com.carrotsearch.randomizedtesting.rules.RamUsageEstimator.createCacheEntry(RamUsageEstimator.java:602)
at 
com.carrotsearch.randomizedtesting.rules.RamUsageEstimator.measureSizeOf(RamUsageEstimator.java:545)
at 
com.carrotsearch.randomizedtesting.rules.RamUsageEstimator.sizeOfAll(RamUsageEstimator.java:387)
at 
com.carrotsearch.randomizedtesting.rules.StaticFieldsInvariantRule$1.afterAlways(StaticFieldsInvariantRule.java:129)
... 10 more
Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make field 
private final byte[] java.lang.String.value accessible: module java.base does 
not "opens java.lang" to unnamed module @101cbf3
at 
java.base/jdk.internal.reflect.Reflection.throwInaccessibleObjectException(Reflection.java:427)
at 
java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:201)
at 
java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:171)
at java.base/java.lang.reflect.Field.setAccessible(Field.java:165)
at 
com.carrotsearch.randomizedtesting.rules.RamUsageEstimator$3.run(RamUsageEstimator.java:597)
at 
com.carrotsearch.randomizedtesting.rules.RamUsageEstimator$3.run(RamUsageEstimator.java:594)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at 
com.carrotsearch.randomizedtesting.rules.RamUsageEstimator.createCacheEntry(RamUsageEstimator.java:594)
... 13 more


FAILED:  
junit.framework.TestSuite.org.apache.lucene.index.TestIndexWriterExceptions

Error Message:
Clean up static fields (in @AfterClass?) and null them, your test still has 
references to classes of which the sizes cannot be measured due to security 
restrictions or Java 9 module encapsulation:   - private static 
java.lang.String 
org.apache.lucene.index.TestIndexWriterExceptions.CRASH_FAIL_MESSAGE

Stack Trace:
junit.framework.AssertionFailedError: Clean up static fields (in @AfterClass?) 
and null them, your test still has references to classes of which the sizes 
cannot be measured due to security restrictions or Java 9 module encapsulation:
  - private static java.lang.String 
org.apache.lucene.index.TestIndexWriterExceptions.CRASH_FAIL_MESSAGE
at __randomizedtesting.SeedInfo.seed([69F8ACEB4E10CEB5]:0)
at 
com.carrotsearch.randomizedtesting.rules.StaticFieldsInvariantRule$1.afterAlways(StaticFieldsInvariantRule.java:146)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at

[jira] [Updated] (LUCENE-7587) New FacetQuery and MultiFacetQuery


 [ 
https://issues.apache.org/jira/browse/LUCENE-7587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Emmanuel Keller updated LUCENE-7587:

Attachment: LUCENE-7587.patch

> New FacetQuery and MultiFacetQuery
> --
>
> Key: LUCENE-7587
> URL: https://issues.apache.org/jira/browse/LUCENE-7587
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: master (7.0), 6.3.1
>Reporter: Emmanuel Keller
>Priority: Minor
>  Labels: facet, faceting
> Fix For: master (7.0), 6.3.1
>
> Attachments: LUCENE-7587.patch
>
>
> This patch introduces two convenient queries: FacetQuery and MultiFacetQuery.
> It can be useful to be able to filter a complex query on one or many facet 
> value.
> - FacetQuery acts as a TermQuery on a FacetField.
> - MultiFacetQuery acts as a MultiTermQuery on a FacetField.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-7587) New FacetQuery and MultiFacetQuery

Emmanuel Keller created LUCENE-7587:
---

 Summary: New FacetQuery and MultiFacetQuery
 Key: LUCENE-7587
 URL: https://issues.apache.org/jira/browse/LUCENE-7587
 Project: Lucene - Core
  Issue Type: Improvement
Affects Versions: master (7.0), 6.3.1
Reporter: Emmanuel Keller
Priority: Minor
 Fix For: master (7.0), 6.3.1


This patch introduces two convenient queries: FacetQuery and MultiFacetQuery.

It can be useful to be able to filter a complex query on one or many facet 
value.

- FacetQuery acts as a TermQuery on a FacetField.
- MultiFacetQuery acts as a MultiTermQuery on a FacetField.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

RE: JDK 9 b148 including a refresh of the module system is available on java.net

Hi Rory,

 

thanks for the message! I put build 148 in the Jenkins test rotation. Let’s see 
what happens.

 

Uwe

 

-

Uwe Schindler

Achterdiek 19, D-28357 Bremen

http://www.thetaphi.de  

eMail: u...@thetaphi.de

 

From: Rory O'Donnell [mailto:rory.odonn...@oracle.com] 
Sent: Friday, December 9, 2016 11:34 AM
To: u...@thetaphi.de
Cc: rory.odonn...@oracle.com; Dalibor Topic ; 
balchandra.vai...@oracle.com; abdul.kolarku...@oracle.com; Dawid Weiss 
; dev@lucene.apache.org
Subject: JDK 9 b148 including a refresh of the module system is available on 
java.net

 

 

Hi Uwe & Dawid, 


JDK 9 build b148   includes an important 
Refresh of the module system [1] , summary of  changes are listed here  
 . 

This refresh includes a disruptive change that is important to understand. 

For those that have been trying out modules with regular JDK 9 builds then be 
aware that `requires public` changes to `requires transitive`. In addition, the 
binary representation of the module declaration (module-info.class) has changed 
so that you need to recompile any modules that were compiled with previous JDK 
9 builds. 

As things stand today in JDK 9 then you use setAccessible to break into 
non-public elements of any type in exported packages. However, it cannot be 
used to break into any type in non-exported package. The current specified 
behavior was a compromise for the initial integration of the module system. It 
is of course not very satisfactory, hence the #AwkwardStrongEncapsulation issue 
[2] on the JSR 376 issues list. With the updated proposal in the JSR, this 
refresh changes setAccessible further so that it cannot be used to break into 
non-public types, or non-public elements of public types, in exported packages. 
Code that uses setAccessible to hack into the private constructor of 
java.lang.invoke.MethodHandles.Lookup will be disappointed for example.

This change will expose hacks in many existing libraries and tools. As a 
workaround then a new command line option `--add-opens` can be used to open 
specific packages for "deep reflection". For example, a really popular build 
tool fails with this refresh because it uses setAccessible + core reflection to 
hack into a private field of an unmodifiable collection so that it can mutate 
it, facepalm! This code will continue to work as before when run with 
`--add-opens java.base/java.util=ALL-UNNAMED` to open the package java.util in 
module java.base to "all unnamed modules" (think class path).

Any help reporting issues to popular tools and libraries would be appreciated. 

A debugging aid that is useful to identify issues is to run with 
-Dsun.reflect.debugModuleAccessChecks=true to get a stack trace when 
setAccessible fails, this is particularly useful when code swallows exceptions 
without any logging.


Rgds,Rory


[1] http://mail.openjdk.java.net/pipermail/jdk9-dev/2016-November/005276.html 

 
[2] 
http://openjdk.java.net/projects/jigsaw/spec/issues/#AwkwardStrongEncapsulation



-- 
Rgds,Rory O'Donnell
Quality Engineering Manager
Oracle EMEA , Dublin, Ireland

[jira] [Commented] (SOLR-9835) Create another replication mode for SolrCloud

2016-12-09 Thread Cao Manh Dat (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-9835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15735547#comment-15735547
]

Cao Manh Dat commented on SOLR-9835:

Here are the detail design that I and Shalin have been working for recently

h2. Overview
The current replication mechanism of SolrCloud is called state machine, which
replicas start in same initial state and for each input, the input is
distributed across replicas so all replicas will end up with same next state.

But this type of replication has some drawbacks :
- The indexing and commits have to run on all replicas
- Slow recovery, because if replica miss more than N updates on its down time,
the replica has to (usually) download the entire index from its leader.

So we introduce another replication for SolrCloud called state transfer, which
acts like master/slave replication. Basically:
- Leader distribute the update to other replicas, but only the leader applies
the update to IndexWriter; other replicas just store the update to UpdateLog
(act like replication)
- Replicas frequently polling the latest segments from leader.

Pros:
- Lightweight for indexing, because only leader are running the updates and
commits
- Very fast recovery. If a replica fails, it just has to download newest
segments, instead of re-downloading entire index.

Cons :
- Leader can become a hotspot when there are many replicas (we can optimize
later)
- Longer turnaround time compared to current NRT replication

h2. Commit
When we commit, we write the version of update command into the commit data in
addition to the timestamp that is written today.

h2. Update
# When a replica receive an update request it will forward the update request
to
#* Correct leader ( in case of add document, delete by id )
#* All leaders ( in case of delete by query, commit )
# Leader assigns version to update as it does today, writes to update log and
applies the update to IndexWriter
# Leader distributes the update to its replicas
# When replica receives an update, it writes the update to update log and
return successfully
# Leader return successful

h2. Periodic Segment Replication
Replica will poll the leader for the latest commit point generation and version
and compare against the latest commit point locally. If it is the same, then
replication is successful and nothing needs to be done. If not, then:
# Replica downloads all files since the local commit point from the leader
# Installs into the index, synchronize on the update log, close the old tlog,
create a new one and copy over all records from the old tlog which have version
greater than the latest commit point’s version. This is to ensure that any
update which hasn’t made it to the index is preserved in the current tlog. This
might lead to duplication of updates in the previous and the current tlogs but
that is okay.
# Replica re-opens the searcher

For example:
Leader has the following tlogs
- TLog1 has versions 1,2,3,commit
- TLog2 has versions 4,5

Before segment replication, the replica have the following tlogs:
- TLog1 - 1,2,3,4,commit

After segment replication, the replica will have the following tlogs:
- TLog1 - 1,2,3,4,commit
- TLog2 - 4,5

During this process, the replica does not need to be put into ‘recovery’ state.
It continues to be ‘active’ and participate in indexing and searches.

h2. Replica recovery
# Replica puts itself in ‘recovery’ state.
# Replica compares its latest commit point against the leader
# If the index is same as leader, then it performs ‘peersync’ with the leader
but only writes the peersync updates to its update log
# If the index is not the same as leader or if peersync fails, the replica:
#* Puts its update log in “buffering” state
#* Issues a hard commit to the leader
#* Copies the index commit points from the leader that do not exist locally
#* Publishes itself as ‘active’
# The “buffering” state in the above steps ensures that any updates that
haven’t been committed to the leader are also present/replicated to the
replica’s current transaction log

With respect to the current recovery strategy in Solr, we need only one change
which is to check the index version of leader vs replica before we attempt a
peersync.

h2. Leader Election
When a leader dies, a candidate replica will become a new leader. The leader
election algorithm remains mostly the same except that after the “sync” step,
the leader candidate will replay its transaction log before publishing itself
as the leader.

h2. Collection property to switch replication scheme
A new property called “onlyLeaderIndexes” will be added to the collection. Any
collection that has this property set to true will only index to the elected
leader and the rest of the replicas will only fetch index segments from the
leader as described above in the document. This property must be set during
collection creation. It will default to “false”. Existing

[jira] [Updated] (LUCENE-7581) IndexWriter#updateDocValues can break index sorting

2016-12-09 Thread Ferenczi Jim (JIRA)


 [ 
https://issues.apache.org/jira/browse/LUCENE-7581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ferenczi Jim updated LUCENE-7581:
-
Attachment: LUCENE-7581.patch

Here is a patch that fails DV updates on a field involved in the index sort.
I also modified TestIndexSorting#testConcurrentDVUpdates which now test DV 
updates that are not involved in the index sort.

> IndexWriter#updateDocValues can break index sorting
> ---
>
> Key: LUCENE-7581
> URL: https://issues.apache.org/jira/browse/LUCENE-7581
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Ferenczi Jim
> Attachments: LUCENE-7581.patch, LUCENE-7581.patch
>
>
> IndexWriter#updateDocValues can break index sorting if it is called on a 
> field that is used in the index sorting specification. 
> TestIndexSorting has a test for this case: #testConcurrentDVUpdates 
> but only L1 merge are checked. Any LN merge would fail the test because the 
> inner sort of the segment is not re-compute during/after DV updates.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 3698 - Unstable!

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3698/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  org.apache.solr.cloud.DocValuesNotIndexedTest.testGroupingDVOnly

Error Message:
Unexpected number of elements in the group for intGSF: 5

Stack Trace:
java.lang.AssertionError: Unexpected number of elements in the group for 
intGSF: 5
at 
__randomizedtesting.SeedInfo.seed([54DB92C93207C57F:CF60FC917F5FF721]:0)
at org.junit.Assert.fail(Assert.java:93)
at 
org.apache.solr.cloud.DocValuesNotIndexedTest.testGroupingDVOnly(DocValuesNotIndexedTest.java:371)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10753 lines...]
   [junit4] Suite: org.apache.solr.cloud.DocValuesNotIndexedTest
   [junit4]   2> Creating dataDir:

[JENKINS] Lucene-Solr-6.x-Linux (64bit/jdk1.8.0_102) - Build # 2378 - Still Unstable!

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/2378/
Java: 64bit/jdk1.8.0_102 -XX:-UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.cloud.DocValuesNotIndexedTest.testGroupingDocAbsent

Error Message:
Did not find the expected number of groups! expected:<4> but was:<3>

Stack Trace:
java.lang.AssertionError: Did not find the expected number of groups! 
expected:<4> but was:<3>
at 
__randomizedtesting.SeedInfo.seed([23A4C1D3ADB37234:FD131A5B48E232F1]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at 
org.apache.solr.cloud.DocValuesNotIndexedTest.testGroupingDocAbsent(DocValuesNotIndexedTest.java:299)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10724 lines...]
   [junit4] Suite:

[jira] [Commented] (SOLR-8542) Integrate Learning to Rank into Solr

2016-12-09 Thread adeppa (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-8542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15735376#comment-15735376
 ] 

adeppa commented on SOLR-8542:
--

Hi Christine


I done some change accordingly 6.3 solr ,i will create different PR and share 
with you ,if you have time please review and validate my changes 


Thanks
Adeppa 

> Integrate Learning to Rank into Solr
> 
>
> Key: SOLR-8542
> URL: https://issues.apache.org/jira/browse/SOLR-8542
> Project: Solr
>  Issue Type: New Feature
>Reporter: Joshua Pantony
>Assignee: Christine Poerschke
>Priority: Minor
> Attachments: SOLR-8542-branch_5x.patch, SOLR-8542-trunk.patch, 
> SOLR-8542.patch
>
>
> This is a ticket to integrate learning to rank machine learning models into 
> Solr. Solr Learning to Rank (LTR) provides a way for you to extract features 
> directly inside Solr for use in training a machine learned model. You can 
> then deploy that model to Solr and use it to rerank your top X search 
> results. This concept was previously [presented by the authors at Lucene/Solr 
> Revolution 
> 2015|http://www.slideshare.net/lucidworks/learning-to-rank-in-solr-presented-by-michael-nilsson-diego-ceccarelli-bloomberg-lp].
> [Read through the 
> README|https://github.com/bloomberg/lucene-solr/tree/master-ltr-plugin-release/solr/contrib/ltr]
>  for a tutorial on using the plugin, in addition to how to train your own 
> external model.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1177 - Still Unstable

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1177/

6 tests failed.
FAILED:  
org.apache.solr.cloud.CdcrReplicationDistributedZkTest.testReplicationAfterLeaderChange

Error Message:
Timeout waiting for CDCR replication to complete @source_collection:shard1

Stack Trace:
java.lang.RuntimeException: Timeout waiting for CDCR replication to complete 
@source_collection:shard1
at 
__randomizedtesting.SeedInfo.seed([9BBC906B20753285:494CDC887EDA94B7]:0)
at 
org.apache.solr.cloud.BaseCdcrDistributedZkTest.waitForReplicationToComplete(BaseCdcrDistributedZkTest.java:795)
at 
org.apache.solr.cloud.CdcrReplicationDistributedZkTest.testReplicationAfterLeaderChange(CdcrReplicationDistributedZkTest.java:305)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at

[jira] [Commented] (SOLR-9824) Documents indexed in bulk are replicated using too many HTTP requests

2016-12-09 Thread David Smiley (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-9824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15735278#comment-15735278
 ] 

David Smiley commented on SOLR-9824:


Ok I get we want a flag to articulate that we want long-running threads.  It's 
really long-running _connections_ specifically what we want; it sounds clearer 
to say it that way IMO.  Maybe longLastingThreads/Connections could be 
synonymous with a poll time of MAX_VALUE?
Some other review comments:
* longLastingThreads (perhaps better renamed longLastingConnections) should be 
final.
* Both places where poll() is invoked, has the same surrounding try-catch 
InterruptedException logic; it could be extracted to a method that returns the 
Update (possibly null).
** say what's the harm in removing the {{if !longLastingThreads}} so that it 
doesn't propagate; it just finishes the connection. That sounds like the right 
behavior to do always.
* I presume you're going to effectively revert SOLR-7333 and all related 
changes?

RE Futures: I get what you're saying but I think it introduces needless 
complexity.  We're conditionally creating a Map, which needs to be populated 
and get items removed under the safety synchronized.  Runners need to be 
submitted differently to get Futures.  I think that's much more complex than 
saving the current Thread to a field on the Runner.

bq. (RE interruption) It's fine - in our case we know all the outstanding 
documents have been added to the queue by the time we are in the 
blockuntilfinished block. We don't access CUSS in a multi threaded manner. Once 
we are in blockUntilFinished and the queue is empty, we know we are just 
interrupting poll.

The spot where blockUntilFinished triggers interruption (via Future.cancel) 
indeed only occurs when the queue is empty (and we can be sure of that, and 
that no new docs will get added.  But the Runner could be in the middle of 
sending the last document it got.  I have no idea if somewhere internal to 
doing that, it might cause the sending of that document to not send and/or 
throw.

> Documents indexed in bulk are replicated using too many HTTP requests
> -
>
> Key: SOLR-9824
> URL: https://issues.apache.org/jira/browse/SOLR-9824
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 6.3
>Reporter: David Smiley
> Attachments: SOLR-9824.patch, SOLR-9824.patch, SOLR-9824.patch
>
>
> This takes awhile to explain; bear with me. While working on bulk indexing 
> small documents, I looked at the logs of my SolrCloud nodes.  I noticed that 
> shards would see an /update log message every ~6ms which is *way* too much.  
> These are requests from one shard (that isn't a leader/replica for these docs 
> but the recipient from my client) to the target shard leader (no additional 
> replicas).  One might ask why I'm not sending docs to the right shard in the 
> first place; I have a reason but it's besides the point -- there's a real 
> Solr perf problem here and this probably applies equally to 
> replicationFactor>1 situations too.  I could turn off the logs but that would 
> hide useful stuff, and it's disconcerting to me that so many short-lived HTTP 
> requests are happening, somehow at the bequest of DistributedUpdateProcessor. 
>  After lots of analysis and debugging and hair pulling, I finally figured it 
> out.  
> In SOLR-7333 ([~tpot]) introduced an optimization called 
> {{UpdateRequest.isLastDocInBatch()}} in which ConcurrentUpdateSolrClient will 
> poll with a '0' timeout to the internal queue, so that it can close the 
> connection without it hanging around any longer than needed.  This part makes 
> sense to me.  Currently the only spot that has the smarts to set this flag is 
> {{JavaBinUpdateRequestCodec.unmarshal.readOuterMostDocIterator()}} at the 
> last document.  So if a shard received docs in a javabin stream (but not 
> other formats) one would expect the _last_ document to have this flag.  
> There's even a test.  Docs without this flag get the default poll time; for 
> javabin it's 25ms.  Okay.
> I _suspect_ that if someone used CloudSolrClient or HttpSolrClient to send 
> javabin data in a batch, the intended efficiencies of SOLR-7333 would apply.  
> I didn't try. In my case, I'm using ConcurrentUpdateSolrClient (and BTW 
> DistributedUpdateProcessor uses CUSC too).  CUSC uses the RequestWriter 
> (defaulting to javabin) to send each document separately without any leading 
> marker or trailing marker.  For the XML format by comparison, there is a 
> leading and trailing marker ( ... ).  Since there's no outer 
> container for the javabin unmarshalling to detect the last document, it marks 
> _every_ document as

[jira] [Commented] (SOLR-8542) Integrate Learning to Rank into Solr

2016-12-09 Thread Christine Poerschke (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-8542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15735258#comment-15735258
 ] 

Christine Poerschke commented on SOLR-8542:
---

done:
 * master commit(s)
 * branch_6x commit(s)

next steps:
 * Solr Reference Guide documentation 
(https://cwiki.apache.org/confluence/display/solr/Internal+-+TODO+List as 
starting point)
 * (to avoid duplication) reduce 
[solr/contrib/ltr/README.md|https://github.com/apache/lucene-solr/blob/master/solr/contrib/ltr/README.md]
 content to point to the appropriate Solr Reference Guide section(s)


> Integrate Learning to Rank into Solr
> 
>
> Key: SOLR-8542
> URL: https://issues.apache.org/jira/browse/SOLR-8542
> Project: Solr
>  Issue Type: New Feature
>Reporter: Joshua Pantony
>Assignee: Christine Poerschke
>Priority: Minor
> Attachments: SOLR-8542-branch_5x.patch, SOLR-8542-trunk.patch, 
> SOLR-8542.patch
>
>
> This is a ticket to integrate learning to rank machine learning models into 
> Solr. Solr Learning to Rank (LTR) provides a way for you to extract features 
> directly inside Solr for use in training a machine learned model. You can 
> then deploy that model to Solr and use it to rerank your top X search 
> results. This concept was previously [presented by the authors at Lucene/Solr 
> Revolution 
> 2015|http://www.slideshare.net/lucidworks/learning-to-rank-in-solr-presented-by-michael-nilsson-diego-ceccarelli-bloomberg-lp].
> [Read through the 
> README|https://github.com/bloomberg/lucene-solr/tree/master-ltr-plugin-release/solr/contrib/ltr]
>  for a tutorial on using the plugin, in addition to how to train your own 
> external model.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8542) Integrate Learning to Rank into Solr

2016-12-09 Thread Christine Poerschke (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-8542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15735251#comment-15735251
 ] 

Christine Poerschke commented on SOLR-8542:
---

Hi Adeppa,

Just a quick note to say that the branch_6x commits below (and specifically the 
"master-to-branch_6x backport changes" commit) might potentially help with the 
compile time errors you describe.

In terms of official Solr 6.3 backporting of the LTR plugin, we do not plan to 
backport to [branch_6_3|https://github.com/apache/lucene-solr/tree/branch_6_3] 
but branch_6x will turn into "Solr 6.4" when the next release happens.

Regards,

Christine

> Integrate Learning to Rank into Solr
> 
>
> Key: SOLR-8542
> URL: https://issues.apache.org/jira/browse/SOLR-8542
> Project: Solr
>  Issue Type: New Feature
>Reporter: Joshua Pantony
>Assignee: Christine Poerschke
>Priority: Minor
> Attachments: SOLR-8542-branch_5x.patch, SOLR-8542-trunk.patch, 
> SOLR-8542.patch
>
>
> This is a ticket to integrate learning to rank machine learning models into 
> Solr. Solr Learning to Rank (LTR) provides a way for you to extract features 
> directly inside Solr for use in training a machine learned model. You can 
> then deploy that model to Solr and use it to rerank your top X search 
> results. This concept was previously [presented by the authors at Lucene/Solr 
> Revolution 
> 2015|http://www.slideshare.net/lucidworks/learning-to-rank-in-solr-presented-by-michael-nilsson-diego-ceccarelli-bloomberg-lp].
> [Read through the 
> README|https://github.com/bloomberg/lucene-solr/tree/master-ltr-plugin-release/solr/contrib/ltr]
>  for a tutorial on using the plugin, in addition to how to train your own 
> external model.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-5944) Support updates of numeric DocValues


[ 
https://issues.apache.org/jira/browse/SOLR-5944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15735034#comment-15735034
 ] 

Ishan Chattopadhyaya edited comment on SOLR-5944 at 12/9/16 11:57 AM:
--

{quote}
that doesn't really answer my question at all: why does it matter if it's a 
dynamic field?
{quote}

If a document's dv field is updated for the first time (i.e. that field never 
existed before in the index before), then the in-place update succeeds if it is 
an explicit field but fails if it is a dynamic field.
It fails with this error: 
{code}
Caused by: java.lang.IllegalArgumentException: can only update existing 
docvalues fields! field=abc_f_dvo, type=NUMERIC
at 
org.apache.lucene.index.IndexWriter.updateDocValues(IndexWriter.java:1715)
at 
org.apache.solr.update.DirectUpdateHandler2.updateDocOrDocValues(DirectUpdateHandler2.java:875)
at 
org.apache.solr.update.DirectUpdateHandler2.doNormalUpdate(DirectUpdateHandler2.java:279)
at 
org.apache.solr.update.DirectUpdateHandler2.addDoc0(DirectUpdateHandler2.java:213)
at 
org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:168)
... 57 more
{code}
The reason for this is that when a document is created, DV fields for all 
explicit fields are created. But when a non-existent dynamic field is attempted 
to be updated subsequently, then the underlying DV field doesn't exist.

Attached a patch to demonstrate this. To demonstrate this, I've disabled the 
schema.isDynamicField() check and also another check that aborts the in-place 
updating for non-existent fields. (Fyi, the latter check is actually erroneous 
and needs to be re-written as I mentioned in the comment 
https://issues.apache.org/jira/browse/SOLR-5944?focusedCommentId=15729798=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15729798)


was (Author: ichattopadhyaya):
{quote}
that doesn't really answer my question at all: why does it matter if it's a 
dynamic field?
{quote}

If a document's dv field is updated for the first time (i.e. that field never 
existed before in the document before), then the in-place update succeeds if it 
is an explicit field but fails if it is a dynamic field.
It fails with this error: 
{code}
Caused by: java.lang.IllegalArgumentException: can only update existing 
docvalues fields! field=abc_f_dvo, type=NUMERIC
at 
org.apache.lucene.index.IndexWriter.updateDocValues(IndexWriter.java:1715)
at 
org.apache.solr.update.DirectUpdateHandler2.updateDocOrDocValues(DirectUpdateHandler2.java:875)
at 
org.apache.solr.update.DirectUpdateHandler2.doNormalUpdate(DirectUpdateHandler2.java:279)
at 
org.apache.solr.update.DirectUpdateHandler2.addDoc0(DirectUpdateHandler2.java:213)
at 
org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:168)
... 57 more
{code}
The reason for this is that when a document is created, DV fields for all 
explicit fields are created. But when a non-existent dynamic field is attempted 
to be updated subsequently, then the underlying DV field doesn't exist.

Attached a patch to demonstrate this. To demonstrate this, I've disabled the 
schema.isDynamicField() check and also another check that aborts the in-place 
updating for non-existent fields. (Fyi, the latter check is actually erroneous 
and needs to be re-written as I mentioned in the comment 
https://issues.apache.org/jira/browse/SOLR-5944?focusedCommentId=15729798=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15729798)

> Support updates of numeric DocValues
> 
>
> Key: SOLR-5944
> URL: https://issues.apache.org/jira/browse/SOLR-5944
> Project: Solr
>  Issue Type: New Feature
>Reporter: Ishan Chattopadhyaya
>Assignee: Shalin Shekhar Mangar
> Attachments: DUP.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,

[jira] [Commented] (LUCENE-7586) fail precommit on varargsArgumentNeedCast

2016-12-09 Thread Christine Poerschke (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-7586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15735145#comment-15735145
 ] 

Christine Poerschke commented on LUCENE-7586:
-

bq. I don't fully understand the issue description. ... - so how does that 
relate to each other?

Apologies, I was trying to provide context for this small patch and caused 
confusion. The logical sequence of things was that I
* saw the Java Warnings (listing and trend chart) on 
https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/
* made [1 line 
change|https://git1-us-west.apache.org/repos/asf?p=lucene-solr.git;a=commit;h=cacabc9a]
 to fix one really easy (varargsArgumentNeedCast) warning listed
* was curious about other varargsArgumentNeedCast warnings in the codebase 
(Jenkins hadn't listed any but I didn't know if all the codebase was covered)
* made change to and tested ecj.javadocs.prefs + org.eclipse.jdt.core.prefs (no 
further varargsArgumentNeedCast warnings found in the codebase)
* created this ticket with attached patch (and wished to mention that Jenkins' 
listing had led to this)

Yes, the extra check is only for Eclipse compiler and setup but still worth 
having?

On a related but separate note an IntelliJ equivalent for Eclipse's 
[org.eclipse.jdt.core.compiler.problem.???=error|https://github.com/apache/lucene-solr/blob/master/dev-tools/eclipse/dot.settings/org.eclipse.jdt.core.prefs#L307-L309]
 might be a nice-to-have.


> fail precommit on varargsArgumentNeedCast
> -
>
> Key: LUCENE-7586
> URL: https://issues.apache.org/jira/browse/LUCENE-7586
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Christine Poerschke
>Assignee: Christine Poerschke
>Priority: Minor
> Attachments: LUCENE-7586.patch
>
>
> Why this, why now?
> I had noticed the Java Warnings (listing and trend chart) on [~thetaphi]'s 
> https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/ and after fixing 
> one really easy warning got curious about others of the same category. There 
> aren't any and so it would seem obvious to update the precommit checks (and 
> Eclipse settings) to prevent future introductions.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-5944) Support updates of numeric DocValues


[ 
https://issues.apache.org/jira/browse/SOLR-5944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15735138#comment-15735138
 ] 

Ishan Chattopadhyaya edited comment on SOLR-5944 at 12/9/16 11:56 AM:
--

{quote}
Specifically: when I tried to remove it, I started seeing NPEs from 
SolrIndexSearcher.decorateDocValueFields in various tests:
{quote}

I tried to chase this bug down, and I think it has led me to some learnings 
regarding the explicit vs dynamic fields (why dynamic fields are any different) 
issue.

{quote}
The reason for this is that when a document is created, DV fields for all 
explicit fields are created. But when a non-existent dynamic field is attempted 
to be updated subsequently, then the underlying DV field doesn't exist.
{quote}
I think I missed the fact that "DV fields for all explicit fields are created" 
*only if* they have a default value set on them. Otherwise, explicit fields are 
no different from dynamic fields (in that they are not created until indexed 
for the first time).

So, therefore, I think we should be checking if the field exists or not, 
irrespective of explicit or dynamic. And if the field doesn't exist, we should 
just delegate to a regular atomic update. Do you think it makes sense?


was (Author: ichattopadhyaya):
{quote}
Specifically: when I tried to remove it, I started seeing NPEs from 
SolrIndexSearcher.decorateDocValueFields in various tests:
{quote}

I tried to chase this bug down, and I think it has led me to some learnings 
regarding the explicit vs dynamic fields (why dynamic fields are any different) 
issue.

{quote}
The reason for this is that when a document is created, DV fields for all 
explicit fields are created. But when a non-existent dynamic field is attempted 
to be updated subsequently, then the underlying DV field doesn't exist.
{quote}
I think I missed the fact that "DV fields for all explicit fields are created" 
*only if* they have a default value set on them. Otherwise, explicit fields are 
no different from dynamic fields.

So, therefore, I think we should be checking if the field exists or not, 
irrespective of explicit or dynamic. And if the field doesn't exist, we should 
just delegate to a regular atomic update. Do you think it makes sense?

> Support updates of numeric DocValues
> 
>
> Key: SOLR-5944
> URL: https://issues.apache.org/jira/browse/SOLR-5944
> Project: Solr
>  Issue Type: New Feature
>Reporter: Ishan Chattopadhyaya
>Assignee: Shalin Shekhar Mangar
> Attachments: DUP.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> TestStressInPlaceUpdates.eb044ac71.beast-167-failure.stdout.txt, 
> TestStressInPlaceUpdates.eb044ac71.beast-587-failure.stdout.txt, 
> TestStressInPlaceUpdates.eb044ac71.failures.tar.gz, defensive-checks.log.gz, 
> demo-why-dynamic-fields-cannot-be-inplace-updated-first-time.patch, 
> hoss.62D328FA1DEA57FD.fail.txt, hoss.62D328FA1DEA57FD.fail2.txt, 
> hoss.62D328FA1DEA57FD.fail3.txt, hoss.D768DD9443A98DC.fail.txt, 
> hoss.D768DD9443A98DC.pass.txt
>
>
> LUCENE-5189 introduced support for updates to numeric docvalues. It would be 
> really nice to have Solr support this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5944) Support updates of numeric DocValues


[ 
https://issues.apache.org/jira/browse/SOLR-5944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15735138#comment-15735138
 ] 

Ishan Chattopadhyaya commented on SOLR-5944:


{quote}
Specifically: when I tried to remove it, I started seeing NPEs from 
SolrIndexSearcher.decorateDocValueFields in various tests:
{quote}

I tried to chase this bug down, and I think it has led me to some learnings 
regarding the explicit vs dynamic fields (why dynamic fields are any different) 
issue.

{quote}
The reason for this is that when a document is created, DV fields for all 
explicit fields are created. But when a non-existent dynamic field is attempted 
to be updated subsequently, then the underlying DV field doesn't exist.
{quote}
I think I missed the fact that "DV fields for all explicit fields are created" 
*only if* they have a default value set on them. Otherwise, explicit fields are 
no different from dynamic fields.

So, therefore, I think we should be checking if the field exists or not, 
irrespective of explicit or dynamic. And if the field doesn't exist, we should 
just delegate to a regular atomic update. Do you think it makes sense?

> Support updates of numeric DocValues
> 
>
> Key: SOLR-5944
> URL: https://issues.apache.org/jira/browse/SOLR-5944
> Project: Solr
>  Issue Type: New Feature
>Reporter: Ishan Chattopadhyaya
>Assignee: Shalin Shekhar Mangar
> Attachments: DUP.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> TestStressInPlaceUpdates.eb044ac71.beast-167-failure.stdout.txt, 
> TestStressInPlaceUpdates.eb044ac71.beast-587-failure.stdout.txt, 
> TestStressInPlaceUpdates.eb044ac71.failures.tar.gz, defensive-checks.log.gz, 
> demo-why-dynamic-fields-cannot-be-inplace-updated-first-time.patch, 
> hoss.62D328FA1DEA57FD.fail.txt, hoss.62D328FA1DEA57FD.fail2.txt, 
> hoss.62D328FA1DEA57FD.fail3.txt, hoss.D768DD9443A98DC.fail.txt, 
> hoss.D768DD9443A98DC.pass.txt
>
>
> LUCENE-5189 introduced support for updates to numeric docvalues. It would be 
> really nice to have Solr support this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5944) Support updates of numeric DocValues


[ 
https://issues.apache.org/jira/browse/SOLR-5944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15735091#comment-15735091
 ] 

Ishan Chattopadhyaya commented on SOLR-5944:


{quote}
In general, i think there's a gap in the DUP logic related to how the results 
of fetchFullUpdateFromLeader are used when replicas never get dependent updates 
{quote}
{quote}
  // nocommit: INNER ELSE -- I'm concerned/confused by this 
code happening here...
  //
  // at this point, we're replacing the data in our existing 
"in-place" update (cmd) so it
  // becomes a normal "add" using the full SolrInputDocument 
fetched from the leader
  //
  // but this if/else is itself is wrapped in a bigger if (grep 
for "OUTER IF" above
  // and "OUTER ELSE" below) where the "else" clause does some 
sanity checking / processing
  // logic for "// non inplace update, i.e. full document 
update" -- all of which is
  // skipped for our modified "cmd"
  //
  // shouldn't the logic in that outer else clause also be 
applied to our
  // "no longer really an in-place" update?
{quote}
I think that the code there should be refactored more nicely. However, as it is 
currently in the branch, I think the same checks within the outer else are 
fulfilled; with the exception of {{checkDeleteByQueries = true}}, which doesn't 
seem to be used anywhere. However, even if the current code (in the branch) 
works correctly, it is very confusing and complicated and I shall try to 
refactor it in such a way that the code in that outer else block gets 
explicitly executed somehow.

> Support updates of numeric DocValues
> 
>
> Key: SOLR-5944
> URL: https://issues.apache.org/jira/browse/SOLR-5944
> Project: Solr
>  Issue Type: New Feature
>Reporter: Ishan Chattopadhyaya
>Assignee: Shalin Shekhar Mangar
> Attachments: DUP.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> TestStressInPlaceUpdates.eb044ac71.beast-167-failure.stdout.txt, 
> TestStressInPlaceUpdates.eb044ac71.beast-587-failure.stdout.txt, 
> TestStressInPlaceUpdates.eb044ac71.failures.tar.gz, defensive-checks.log.gz, 
> demo-why-dynamic-fields-cannot-be-inplace-updated-first-time.patch, 
> hoss.62D328FA1DEA57FD.fail.txt, hoss.62D328FA1DEA57FD.fail2.txt, 
> hoss.62D328FA1DEA57FD.fail3.txt, hoss.D768DD9443A98DC.fail.txt, 
> hoss.D768DD9443A98DC.pass.txt
>
>
> LUCENE-5189 introduced support for updates to numeric docvalues. It would be 
> really nice to have Solr support this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5944) Support updates of numeric DocValues


[ 
https://issues.apache.org/jira/browse/SOLR-5944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15735064#comment-15735064
 ] 

Ishan Chattopadhyaya commented on SOLR-5944:


{quote}
The concern i have is that AFAICT rebuilding the Arrays.asList(tlog, 
prevMapLog, prevMapLog2) in every iteration of the while loop seems to be at 
cross purposes with the reason why the while loop might have more then one 
iteration.
{quote}
Your explanation makes complete sense. I think we should pull it out. I've yet 
to look deeply into your suggestion of having it all within the same 
synchronized block, but it makes sense to me.

> Support updates of numeric DocValues
> 
>
> Key: SOLR-5944
> URL: https://issues.apache.org/jira/browse/SOLR-5944
> Project: Solr
>  Issue Type: New Feature
>Reporter: Ishan Chattopadhyaya
>Assignee: Shalin Shekhar Mangar
> Attachments: DUP.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> TestStressInPlaceUpdates.eb044ac71.beast-167-failure.stdout.txt, 
> TestStressInPlaceUpdates.eb044ac71.beast-587-failure.stdout.txt, 
> TestStressInPlaceUpdates.eb044ac71.failures.tar.gz, defensive-checks.log.gz, 
> demo-why-dynamic-fields-cannot-be-inplace-updated-first-time.patch, 
> hoss.62D328FA1DEA57FD.fail.txt, hoss.62D328FA1DEA57FD.fail2.txt, 
> hoss.62D328FA1DEA57FD.fail3.txt, hoss.D768DD9443A98DC.fail.txt, 
> hoss.D768DD9443A98DC.pass.txt
>
>
> LUCENE-5189 introduced support for updates to numeric docvalues. It would be 
> really nice to have Solr support this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5944) Support updates of numeric DocValues


 [ 
https://issues.apache.org/jira/browse/SOLR-5944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ishan Chattopadhyaya updated SOLR-5944:
---
Attachment: 
demo-why-dynamic-fields-cannot-be-inplace-updated-first-time.patch

{quote}
that doesn't really answer my question at all: why does it matter if it's a 
dynamic field?
{quote}

If a document's dv field is updated for the first time (i.e. that field never 
existed before in the document before), then the in-place update succeeds if it 
is an explicit field but fails if it is a dynamic field.
It fails with this error: 
{code}
Caused by: java.lang.IllegalArgumentException: can only update existing 
docvalues fields! field=abc_f_dvo, type=NUMERIC
at 
org.apache.lucene.index.IndexWriter.updateDocValues(IndexWriter.java:1715)
at 
org.apache.solr.update.DirectUpdateHandler2.updateDocOrDocValues(DirectUpdateHandler2.java:875)
at 
org.apache.solr.update.DirectUpdateHandler2.doNormalUpdate(DirectUpdateHandler2.java:279)
at 
org.apache.solr.update.DirectUpdateHandler2.addDoc0(DirectUpdateHandler2.java:213)
at 
org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:168)
... 57 more
{code}
The reason for this is that when a document is created, DV fields for all 
explicit fields are created. But when a non-existent dynamic field is attempted 
to be updated subsequently, then the underlying DV field doesn't exist.

Attached a patch to demonstrate this. To demonstrate this, I've disabled the 
schema.isDynamicField() check and also another check that aborts the in-place 
updating for non-existent fields. (Fyi, the latter check is actually erroneous 
and needs to be re-written as I mentioned in the comment 
https://issues.apache.org/jira/browse/SOLR-5944?focusedCommentId=15729798=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15729798)

> Support updates of numeric DocValues
> 
>
> Key: SOLR-5944
> URL: https://issues.apache.org/jira/browse/SOLR-5944
> Project: Solr
>  Issue Type: New Feature
>Reporter: Ishan Chattopadhyaya
>Assignee: Shalin Shekhar Mangar
> Attachments: DUP.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> TestStressInPlaceUpdates.eb044ac71.beast-167-failure.stdout.txt, 
> TestStressInPlaceUpdates.eb044ac71.beast-587-failure.stdout.txt, 
> TestStressInPlaceUpdates.eb044ac71.failures.tar.gz, defensive-checks.log.gz, 
> demo-why-dynamic-fields-cannot-be-inplace-updated-first-time.patch, 
> hoss.62D328FA1DEA57FD.fail.txt, hoss.62D328FA1DEA57FD.fail2.txt, 
> hoss.62D328FA1DEA57FD.fail3.txt, hoss.D768DD9443A98DC.fail.txt, 
> hoss.D768DD9443A98DC.pass.txt
>
>
> LUCENE-5189 introduced support for updates to numeric docvalues. It would be 
> really nice to have Solr support this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Windows (64bit/jdk1.8.0_102) - Build # 6275 - Still unstable!

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/6275/
Java: 64bit/jdk1.8.0_102 -XX:+UseCompressedOops -XX:+UseG1GC

2 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.lucene.store.TestNIOFSDirectory

Error Message:
Could not remove the following files (in the order of attempts):
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\core\test\J0\temp\lucene.store.TestNIOFSDirectory_25572201FDEDA61F-001\testThreadSafety-001:
 java.nio.file.AccessDeniedException: 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\core\test\J0\temp\lucene.store.TestNIOFSDirectory_25572201FDEDA61F-001\testThreadSafety-001

C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\core\test\J0\temp\lucene.store.TestNIOFSDirectory_25572201FDEDA61F-001:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\core\test\J0\temp\lucene.store.TestNIOFSDirectory_25572201FDEDA61F-001
 

Stack Trace:
java.io.IOException: Could not remove the following files (in the order of 
attempts):
   
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\core\test\J0\temp\lucene.store.TestNIOFSDirectory_25572201FDEDA61F-001\testThreadSafety-001:
 java.nio.file.AccessDeniedException: 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\core\test\J0\temp\lucene.store.TestNIOFSDirectory_25572201FDEDA61F-001\testThreadSafety-001
   
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\core\test\J0\temp\lucene.store.TestNIOFSDirectory_25572201FDEDA61F-001:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\core\test\J0\temp\lucene.store.TestNIOFSDirectory_25572201FDEDA61F-001

at __randomizedtesting.SeedInfo.seed([25572201FDEDA61F]:0)
at org.apache.lucene.util.IOUtils.rm(IOUtils.java:323)
at 
org.apache.lucene.util.TestRuleTemporaryFilesCleanup.afterAlways(TestRuleTemporaryFilesCleanup.java:216)
at 
com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.afterAlways(TestRuleAdapter.java:31)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)


FAILED:  
org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.testDelegationTokenCancelFail

Error Message:
expected:<200> but was:<404>

Stack Trace:
java.lang.AssertionError: expected:<200> but was:<404>
at 
__randomizedtesting.SeedInfo.seed([C67FF28A90D35605:AEC0C7A0404944E9]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at 
org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.cancelDelegationToken(TestSolrCloudWithDelegationTokens.java:140)
at 
org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.testDelegationTokenCancelFail(TestSolrCloudWithDelegationTokens.java:294)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at

[JENKINS] Lucene-Solr-6.x-Linux (64bit/jdk1.8.0_102) - Build # 2377 - Unstable!

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/2377/
Java: 64bit/jdk1.8.0_102 -XX:-UseCompressedOops -XX:+UseSerialGC

3 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestCoreDiscovery

Error Message:
ObjectTracker found 5 object(s) that were not released!!! 
[MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, 
MDCAwareThreadPoolExecutor] 
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException  at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
  at 
org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
  at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:332)  at 
org.apache.solr.core.SolrCore.initIndex(SolrCore.java:641)  at 
org.apache.solr.core.SolrCore.(SolrCore.java:847)  at 
org.apache.solr.core.SolrCore.(SolrCore.java:775)  at 
org.apache.solr.core.CoreContainer.create(CoreContainer.java:842)  at 
org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1113)  at 
org.apache.solr.core.TestCoreDiscovery.testTooManyTransientCores(TestCoreDiscovery.java:211)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)  at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)  
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:498)  at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
  at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
  at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
  at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
  at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
  at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
  at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
  at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
  at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
  at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
  at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
  at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
  at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
  at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
  at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
  at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
  at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
  at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
  at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
  at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
  at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
  at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
  at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
  at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
  at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
  at java.lang.Thread.run(Thread.java:745)

JDK 9 b148 including a refresh of the module system is available on java.net

2016-12-09 Thread Rory O'Donnell



Hi Uwe & Dawid,


JDK 9 build b148  includes an important 
Refresh of the module system [1] , summary of changes are listed here 
.


*This refresh includes a disruptive change that is important to understand.

*For those that have been trying out modules with regular JDK 9 builds 
then be aware that `requires public` changes to `requires transitive`. 
In addition, the binary representation of the module declaration 
(module-info.class) has changed so that you need to recompile any 
modules that were compiled with previous JDK 9 builds.


As things stand today in JDK 9 then you use setAccessible to break into 
non-public elements of any type in exported packages. However, it cannot 
be used to break into any type in non-exported package. The current 
specified behavior was a compromise for the initial integration of the 
module system. It is of course not very satisfactory, hence the 
#AwkwardStrongEncapsulation issue [2] on the JSR 376 issues list. With 
the updated proposal in the JSR, this refresh changes setAccessible 
further so that it cannot be used to break into non-public types, or 
non-public elements of public types, in exported packages. Code that 
uses setAccessible to hack into the private constructor of 
java.lang.invoke.MethodHandles.Lookup will be disappointed for example.


This change will expose hacks in many existing libraries and tools. As a 
workaround then a new command line option `--add-opens` can be used to 
open specific packages for "deep reflection". For example, a really 
popular build tool fails with this refresh because it uses setAccessible 
+ core reflection to hack into a private field of an unmodifiable 
collection so that it can mutate it, facepalm! This code will continue 
to work as before when run with `--add-opens 
java.base/java.util=ALL-UNNAMED` to open the package java.util in module 
java.base to "all unnamed modules" (think class path).


*Any help reporting issues to popular tools and libraries would be 
appreciated. *


A debugging aid that is useful to identify issues is to run with 
-Dsun.reflect.debugModuleAccessChecks=true to get a stack trace when 
setAccessible fails, this is particularly useful when code swallows 
exceptions without any logging.



Rgds,Rory


[1] 
http://mail.openjdk.java.net/pipermail/jdk9-dev/2016-November/005276.html 

[2] 
http://openjdk.java.net/projects/jigsaw/spec/issues/#AwkwardStrongEncapsulation


--
Rgds,Rory O'Donnell
Quality Engineering Manager
Oracle EMEA , Dublin, Ireland

[JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - Build # 18480 - Still Unstable!