Jenkins Test Reports "outage" the past few days

[Chris Hostetter]

(note: cross posted, if you have a reason to reply, please consider your 
reply-to accordingly)


TL;DR: IF you use my jenknins stats, and notice a bunch of new failures 
today, they may be from commits earlier this week.


http://fucit.org/solr-jenkins-reports/

Just wanted to put out a quick note that my jenkins test failure report 
system was malfunctioning from some time late monday (in the US) when my 
hosting provider upgraded the OS (breaking some shared library assumptions 
i didn't realize i had).


The feed scraping was still working, so when i did a quick check of the 
site the "last updated" and list of recent jobs still looked correct, but 
it wasn't until yesterday that i realized it wasn't extracting the test 
data from each job correctly -- but it was updating the aggregate stats 
file, as if there were no new failures -- so all the stats were low.


I fixed it ~18 hours ago, and they way it's setup it was able to 
"backfill" *SOME* of the failed test info from the last few days (as long 
as the jenkins job info hadn't been pruned by the source server) but in 
general the "Past 7 days" metrics are going to be off for a few days, and 
the "Past 24 hours" won't be 100% accurate for a few more hours.


Most importantly though: If you look at the reports at any point this past 
week and then looked at it again atoday and think "Holy shit, something 
just changed that caused a bunch of new failures!" ... that "Something" 
isn't neccessarily from the past 24 hours ... it could have been much 
earlier in the week, and the stats just didn't reflect it until now.




-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Ascii folding

[Chris Hostetter]

: Here's the unicode letter after "th":
: https://www.fileformat.info/info/unicode/char/0435/index.htm
: 
: To my surprise, I couldn't find it in the ascii folding filter:
: 
: 
https://github.com/apache/lucene/blob/main/lucene/analysis/common/src/java/org/apache/lucene/analysis/miscellaneous/ASCIIFoldingFilter.java
: 
: Anybody remembers whether the omission of Cyrillic characters was
: intentional (there is quite a few of them that are nearly identical in
: appearance to Latin letters).

>From the javadocs, i'm going to guess it's because the the filter focuses 
on "Latin_characters_in_Unicode" ... and your "CYRILLIC SMALL LETTER IE" 
isn't described as being a "(adjective) LATIN noun (WITH noun)" like all 
of the other characters that are considered to have a direct mapping to 
the "ASCII" / latin characters.

If you look back at when it was added...

https://issues.apache.org/jira/browse/LUCENE-1390

...the original focus was on deprecating "ISOLatin1AccentFilter" and 
replacing it with "a more comprehensive version of this code that included 
not just ISO-Latin-1 (ISO-8859-1) but the entire Latin 1 and Latin 
Extended A unicode blocks."  (The originally proposed name was 
'ISOLatinAccentFilter') ... subsequent discussion focused on adding more 
Latin blocks.

There was a related issue at the time which initially aimed to add a 
more general "UnicodeNormalizationFilter" that ultimated resulted in 
adding the "ICU" analysis classes...

https://issues.apache.org/jira/browse/LUCENE-1343

..which IIUC may better handle "CYRILLIC SMALL LETTER IE" (but i haven't 
tested that)



-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene » Lucene-Solr-repro - Build # 1164 - Still Failing!

[Chris Hostetter]

: I'm not even sure how it's possible but this build seems to be using stale
: configuration (repo pointing at the commit that informs about project split
: between Solr and Lucene). I've disabled the plan.

This jenkins job stoped sending emails  ~ March of 2021.

There is a thread from March of 2022 where Smiley mentioned it 
didn't survive the gradle migration, and suggested bring it back...

https://lists.apache.org/list?d...@solr.apache.org:gte=365d:reproduceJenkinsFailures.py


But around Oct 27, it started running again and sending emails ... possily 
because of something going horribly wrong on the jenkins machines 
(possibly related to the other git related failures that started happening 
in other jobs recently?)



-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene » Lucene-NightlyTests-main - Build # 1171 - Unstable!

[Chris Hostetter]

: Seems like an unclean build env issue.

It's actually pretty trivial to reproduce this problem locally...

git co 11436a848cbcc8302b31ca01e63e57dd35a93b1e
./gradlew clean classes testClasses 
git co 361f0a4524d93fc1a793aa72bfcc91c00fa3599e 
./gradlew test --tests TestFlattenGraphFilter


The problem is that jenkins doesn't run "clean" but gradle won't 
(by default) recompile a source set (like "analysis tests" unless it has 
changed -- even when it's dependencies have changed.

In this case, :lucene:core's definition of Automata.makeStringUnion 
changed, but a lot of the code that *calls* that method didn't have to 
change (at a java level)because the method signature's are back compatable 
in terms of compilation -- but not in terms of byte code.

So gradle doesn't think any subprojects that have code that 
calls that method need to be recompiled -- so it doesn't bother...

https://ci-builds.apache.org/job/Lucene/job/Lucene-NightlyTests-main/1171/consoleText

> Task :lucene:analysis.tests:compileTestJava UP-TO-DATE
> Task :lucene:analysis.tests:processTestResources UP-TO-DATE
> Task :lucene:analysis.tests:testClasses UP-TO-DATE
...
> Task :lucene:highlighter:compileJava UP-TO-DATE
> Task :lucene:highlighter:processResources NO-SOURCE
> Task :lucene:highlighter:classes UP-TO-DATE
...
> Task :lucene:highlighter:compileTestJava UP-TO-DATE
> Task :lucene:highlighter:processTestResources UP-TO-DATE
> Task :lucene:highlighter:testClasses UP-TO-DATE
...etc.











: 
: Mike
: 
: On Tue, Oct 31, 2023 at 11:15 AM Apache Jenkins Server <
: jenk...@builds.apache.org> wrote:
: 
: > Build:
: > https://ci-builds.apache.org/job/Lucene/job/Lucene-NightlyTests-main/1171/
: >
: > 88 tests failed.
: > FAILED:
: > org.apache.lucene.analysis.core.TestFlattenGraphFilter.testPathsNotLost
: >
: > Error Message:
: > java.lang.NoSuchMethodError: 'org.apache.lucene.util.automaton.Automaton
: > 
org.apache.lucene.util.automaton.Automata.makeStringUnion(java.util.Collection)'
: >
: > Stack Trace:
: > java.lang.NoSuchMethodError: 'org.apache.lucene.util.automaton.Automaton
: > 
org.apache.lucene.util.automaton.Automata.makeStringUnion(java.util.Collection)'
: > at
: > __randomizedtesting.SeedInfo.seed([18019D3C4539B4C7:1F664550C7327759]:0)
: > at
: > 
org.apache.lucene.analysis.core.TestFlattenGraphFilter.testPathsNotLost(TestFlattenGraphFilter.java:784)
: > at
: > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
: > Method)
: > at
: > 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
: > at
: > 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
: > at java.base/java.lang.reflect.Method.invoke(Method.java:568)
: > at randomizedtesting.runner@2.8.1
: > 
/com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
: > at randomizedtesting.runner@2.8.1
: > 
/com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
: > at randomizedtesting.runner@2.8.1
: > 
/com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
: > at randomizedtesting.runner@2.8.1
: > 
/com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
: > at org.apache.lucene.test_framework@10.0.0-SNAPSHOT
: > 
/org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
: > at org.apache.lucene.test_framework@10.0.0-SNAPSHOT
: > 
/org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
: > at org.apache.lucene.test_framework@10.0.0-SNAPSHOT
: > 
/org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
: > at org.apache.lucene.test_framework@10.0.0-SNAPSHOT
: > 
/org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
: > at org.apache.lucene.test_framework@10.0.0-SNAPSHOT
: > 
/org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
: > at junit@4.13.1
: > /org.junit.rules.RunRules.evaluate(RunRules.java:20)
: > at randomizedtesting.runner@2.8.1
: > 
/com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: > at randomizedtesting.runner@2.8.1
: > 
/com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
: > at randomizedtesting.runner@2.8.1
: > 
/com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
: > at randomizedtesting.runner@2.8.1
: > 
/com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
: > at randomizedtesting.runner@2.8.1
: > 

Re: ByteBufferIndexInput.alreadyClosed creates an exception that doesn't track its cause

[Chris Hostetter]

Uwe: In your PR, you should add these details to the javadocs of 
ByteBufferIndexInput.alreadyClosed(), so future code spelunkers understand 
the choice being made here is intentional :)

: please don't add the NPE here as cause (except for debugging). The NPE is only
: catched to NOT add extra checks in the highly performance sensitive code.
: Actually the NPE is catched to detect the case where the bytebuffer was
: already unset to trigger the already closed. The code uses setting the buffers
: to NULL to signal cause, but it does NOT add a NULL check everywhere. This
: allows Hotspot to compile this code without any bounds checks and signal the
: AlreadyClosedException only when a NPE happens. Adding the NPE as cause would



-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: ByteBufferIndexInput.alreadyClosed creates an exception that doesn't track its cause

[Chris Hostetter]

FWIW: The choice to ignore the original exception goes back to here...

https://issues.apache.org/jira/browse/LUCENE-3588

...circa 2011, where it was focused on catching NPE and throwing 
AlreadyClosedException instead, w/o any particular discussion as to why to 
throw away the original NPE.

If i had to guess it's simply because at that time AlreadyClosedException 
didn't support wrapping any other Throwable.  That wasn't added until 
LUCENE-5958 (circa 2014) which was focused on making sure "tragic" errors 
kept a record of what caused the tragedy, and then include that as the 
'cause' of the AlreadyClosedExceptions throw by 'ensureOpen()'

There didn't seem to be any discussion at that time about reviewing other 
code that might be throwing AlreadyClosedException from a 'catch' block 
that could also be updated to include the cause.

I'd say open a PR to review & update all code that results in 
AlreadyClosedException originating from a catch block?



: Date: Tue, 17 Oct 2023 11:24:03 -0400
: From: Michael Sokolov 
: Reply-To: dev@lucene.apache.org
: To: Lucene Dev 
: Subject: ByteBufferIndexInput.alreadyClosed creates an exception that doesn't
: track its cause
: 
: I was messing around with something that was resulting in
: AlreadyClosedException being thrown and I noticed that we weren't
: tracking the exception that caused it. I found this in
: ByteBufferIndexInput:
: 
:// the unused parameter is just to silence javac about unused variables
:AlreadyClosedException alreadyClosed(RuntimeException unused) {
: -return new AlreadyClosedException("Already closed: " + this);
: +return new AlreadyClosedException("Already closed: " + this, unused);
:}
: 
: and added the cause there, which helped me find and fix my wicked
: ways. Is there a reason we decided not to wrap the "unused"
: RuntimeException there?
: 
: -
: To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
: For additional commands, e-mail: dev-h...@lucene.apache.org
: 
: 

-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene » Lucene-Check-main - Build # 10345 - Unstable!

[Chris Hostetter]

https://github.com/apache/lucene/pull/12382#issuecomment-1744028622

: Date: Thu, 28 Sep 2023 20:08:08 + (UTC)
: From: Apache Jenkins Server 
: Reply-To: dev@lucene.apache.org
: To: bui...@lucene.apache.org
: Subject: [JENKINS] Lucene » Lucene-Check-main - Build # 10345 - Unstable!
: 
: Build: https://ci-builds.apache.org/job/Lucene/job/Lucene-Check-main/10345/
: 
: 1 tests failed.
: FAILED:  org.apache.lucene.search.TestBlockMaxConjunction.testRandom
: 
: Error Message:
: java.lang.AssertionError: Hit 0 docnumbers don't match
: Hits length1=1length2=1
: hit=0: doc4=9.615059 shardIndex=-1,doc1215=9.615059 shardIndex=-1
: for query:+foo:9 +foo:10 +foo:11 +foo:12 +foo:13
: 
: Stack Trace:
: java.lang.AssertionError: Hit 0 docnumbers don't match
: Hits length1=1length2=1
: hit=0: doc4=9.615059 shardIndex=-1,doc1215=9.615059 shardIndex=-1
: for query:+foo:9 +foo:10 +foo:11 +foo:12 +foo:13
:   at 
__randomizedtesting.SeedInfo.seed([C738B747E3320853:B57492485252BE20]:0)
:   at org.junit.Assert.fail(Assert.java:89)
:   at 
org.apache.lucene.tests.search.CheckHits.checkEqual(CheckHits.java:229)
:   at 
org.apache.lucene.tests.search.CheckHits.doCheckTopScores(CheckHits.java:709)
:   at 
org.apache.lucene.tests.search.CheckHits.checkTopScores(CheckHits.java:694)
:   at 
org.apache.lucene.search.TestBlockMaxConjunction.testRandom(TestBlockMaxConjunction.java:81)
:   at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
:   at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
:   at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
:   at java.base/java.lang.reflect.Method.invoke(Method.java:568)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
:   at 
org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
:   at 
org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
:   at 
org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
:   at 
org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
:   at 
org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
:   at org.junit.rules.RunRules.evaluate(RunRules.java:20)
:   at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
:   at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
:   at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
:   at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
:   at 
org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
:   at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
:   at 
org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
:   at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
:   at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
:   at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
:   at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
:   at 
org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
:   at 
org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
:   at 
org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
:   at 
org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
:   at 

Re: [JENKINS] Lucene » Lucene-NightlyTests-main - Build # 1019 - Unstable!

[Chris Hostetter]

: This reproduces constantly for me, I even rolls back to some (random) old
: commit: 9acc6539959 (in last September) and it still reproduces.

FWIW: The oldest known failure of TestGroupFacetCollector.testRandom in my 
archive is the week of 2021-05-31



: Found another similar failure:
: https://ci-builds.apache.org/job/Lucene/job/Lucene-Check-main/8835/
: The error seems to be a more common one but somehow (so far) it only
: happens inside this TestGroupFacetCollector test...
: 
: On Wed, May 31, 2023 at 10:18 PM Apache Jenkins Server <
: jenk...@builds.apache.org> wrote:
: 
: > Build:
: > https://ci-builds.apache.org/job/Lucene/job/Lucene-NightlyTests-main/1019/
: >
: > 1 tests failed.
: > FAILED:
: > org.apache.lucene.search.grouping.TestGroupFacetCollector.testRandom
: >
: > Error Message:
: > java.lang.IndexOutOfBoundsException: Range [0, 0 + 692) out of bounds for
: > length 80
: >
: > Stack Trace:
: > java.lang.IndexOutOfBoundsException: Range [0, 0 + 692) out of bounds for
: > length 80
: > at
: > __randomizedtesting.SeedInfo.seed([46C657C9C98115E0:348A72C678E1A393]:0)
: > at
: > java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:64)
: > at
: > 
java.base/jdk.internal.util.Preconditions.outOfBoundsCheckFromIndexSize(Preconditions.java:82)
: > at
: > 
java.base/jdk.internal.util.Preconditions.checkFromIndexSize(Preconditions.java:361)
: > at java.base/java.util.Objects.checkFromIndexSize(Objects.java:413)
: > at java.base/java.nio.HeapByteBuffer.get(HeapByteBuffer.java:180)
: > at org.apache.lucene.core@10.0.0-SNAPSHOT
: > 
/org.apache.lucene.store.ByteBuffersDataInput.readBytes(ByteBuffersDataInput.java:159)
: > at org.apache.lucene.core@10.0.0-SNAPSHOT
: > 
/org.apache.lucene.store.ByteBuffersIndexInput.readBytes(ByteBuffersIndexInput.java:85)
: > at org.apache.lucene.test_framework@10.0.0-SNAPSHOT
: > 
/org.apache.lucene.tests.store.MockIndexInputWrapper.readBytes(MockIndexInputWrapper.java:148)
: > at org.apache.lucene.core@10.0.0-SNAPSHOT
: > 
/org.apache.lucene.codecs.lucene90.Lucene90DocValuesProducer$TermsDict.decompressBlock(Lucene90DocValuesProducer.java:1234)
: > at org.apache.lucene.core@10.0.0-SNAPSHOT
: > 
/org.apache.lucene.codecs.lucene90.Lucene90DocValuesProducer$TermsDict.next(Lucene90DocValuesProducer.java:1092)
: > at
: > 
org.apache.lucene.search.grouping.TermGroupFacetCollector$MV$SegmentResult.nextTerm(TermGroupFacetCollector.java:437)
: > at
: > 
org.apache.lucene.search.grouping.GroupFacetCollector.mergeSegmentResults(GroupFacetCollector.java:97)
: > at
: > 
org.apache.lucene.search.grouping.TestGroupFacetCollector.testRandom(TestGroupFacetCollector.java:429)
: > at
: > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
: > Method)
: > at
: > 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
: > at
: > 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
: > at java.base/java.lang.reflect.Method.invoke(Method.java:568)
: > at randomizedtesting.runner@2.8.1
: > 
/com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
: > at randomizedtesting.runner@2.8.1
: > 
/com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
: > at randomizedtesting.runner@2.8.1
: > 
/com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
: > at randomizedtesting.runner@2.8.1
: > 
/com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
: > at org.apache.lucene.test_framework@10.0.0-SNAPSHOT
: > 
/org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
: > at org.apache.lucene.test_framework@10.0.0-SNAPSHOT
: > 
/org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
: > at org.apache.lucene.test_framework@10.0.0-SNAPSHOT
: > 
/org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
: > at org.apache.lucene.test_framework@10.0.0-SNAPSHOT
: > 
/org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
: > at org.apache.lucene.test_framework@10.0.0-SNAPSHOT
: > 
/org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
: > at junit@4.13.1
: > /org.junit.rules.RunRules.evaluate(RunRules.java:20)
: > at randomizedtesting.runner@2.8.1
: > 
/com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: > at randomizedtesting.runner@2.8.1
: > 
/com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
: 

Re: [JENKINS] Lucene-9.x-MacOSX (64bit/hotspot/jdk-16.0.2) - Build # 2385 - Unstable!

[Chris Hostetter]

FWIW: This same exact failure (same assertion msg) happened on main 
recently as well.

I've included some log snippets shouwing randomization details of both 
failures below.

Historically this test (with this name) has existed unmodified since long 
before i started keeping records of failure rates, but it has had failure 
"blips" ~14 times since 2021-04-12 -- prio to that, i have no record of it 
*EVER* failing.

Looking at the log snippets, the few things that jump out at me as being 
"common" between them...

 - java >= 16
 - -XX:+UseCompressedOops
 - codec=SimpleText
 - sim=Asserting(RandomSimilarity(queryNorm=false)
 - common "All tests run in this JVM" ...
   - TestOmitPositions
   - TestStopFilter
   - TestXYShape
   - TestAddIndexes
   - TestIndexCommit
   - TestIndexingSequenceNumbers
   - TestDocIDMerger
   - TestLatLonMultiLineShapeQueries
   - TestLucene90StoredFieldsFormatHighCompression



https://jenkins.thetaphi.de/view/Lucene/job/Lucene-main-Linux/42271/
Checking out Revision 72b91156f3d2049e66a987b9cc435c22d6c9e3af 
(refs/remotes/origin/main)
[description-setter] Description set: Java: 64bit/hotspot/jdk-17.0.5 
-XX:+UseCompressedOops -XX:+UseParallelGC

  2> NOTE: reproduce with: gradlew test --tests 
TestSizeBoundedForceMerge.testByteSizeLimit -Dtests.seed=AE682DEBC10C54AC 
-Dtests.multiplier=3 -Dtests.locale=so-KE -Dtests.timezone=Asia/Kathmandu 
-Dtests.asserts=true -Dtests.file.encoding=UTF-8
  2> NOTE: test params are: codec=SimpleText, 
sim=Asserting(RandomSimilarity(queryNorm=false): {}), locale=so-KE, 
timezone=Asia/Kathmandu
  2> NOTE: Linux 5.19.0-42-generic amd64/Eclipse Adoptium 17.0.5 
(64-bit)/cpus=16,threads=1,free=191739968,total=335020032
  2> NOTE: All tests run in this JVM: [TestDemo, TestAnalysisSPILoader, 
TestCharFilter, TestStopFilter, TestPackedTokenAttributeImpl, 
TestDeflateWithPresetDictCompressionMode, TestIndexedDISI, 
TestLucene90NormsFormat, TestLucene90StoredFieldsFormatHighCompression, 
TestStoredFieldsInt, TestBinaryDocument, TestFeatureSort, 
TestKeywordField, TestLatLonMultiLineShapeQueries, 
TestLatLonPointDistanceSort, TestLatLonShape, TestPerFieldConsistency, 
TestXYMultiPointShapeQueries, TestXYShape, TestLine2D, 
TestSimpleWKTShapeParsing, TestXYRectangle, Test2BPostings, 
TestAddIndexes, TestApproximatePriorityQueue, TestBufferedUpdates, 
TestConcurrentMergeScheduler, TestDeletionPolicy, TestDocIDMerger, 
TestDocsWithFieldSet, TestDuelingCodecsAtNight, TestFieldUpdatesBuffer, 
TestFilterMergePolicy, TestIndexCommit, TestIndexSorting, 
TestIndexWriterExceptions, TestIndexWriterMergePolicy, 
TestIndexWriterOutOfFileDescriptors, TestIndexingSequenceNumbers, 
TestLogMergePolicy, TestMergeRateLimiter, TestMultiTermsEnum, 
TestNoDeletionPolicy, TestOmitPositions, TestParallelReaderEmptyIndex, 
TestPerSegmentDeletes, TestReaderClosed, TestSegmentInfos, 
TestSizeBoundedForceMerge]



The details from the 9.x build were...

https://jenkins.thetaphi.de/view/Lucene/job/Lucene-9.x-MacOSX/2385/
Checking out Revision d96e4ffbb5ae1ef0b599c1a83fdba8cb486dfb65 
(refs/remotes/origin/branch_9x)
[description-setter] Description set: Java: 64bit/hotspot/jdk-16.0.2 
-XX:+UseCompressedOops -XX:+UseShenandoahGC

  2> NOTE: reproduce with: gradlew test --tests 
TestSizeBoundedForceMerge.testByteSizeLimit -Dtests.seed=1BBFFFEF44E9DD39 
-Dtests.locale=bs-Cyrl-BA -Dtests.timezone=America/Thunder_Bay 
-Dtests.asserts=true -Dtests.file.encoding=UTF-8
  2> NOTE: test params are: codec=SimpleText, 
sim=Asserting(RandomSimilarity(queryNorm=false): {}), locale=bs-Cyrl-BA, 
timezone=America/Thunder_Bay
  2> NOTE: Mac OS X 10.14.6 x86_64/Eclipse Foundation 16.0.2 
(64-bit)/cpus=6,threads=1,free=156085320,total=471072768
  2> NOTE: All tests run in this JVM: [TestSearch, 
TestAutomatonToTokenStream, TestCharacterUtils, TestStopFilter, 
TestCharTermAttributeImpl, TestCompetitiveFreqNormAccumulator, 
TestHighCompressionMode, TestLucene90DocValuesFormat, 
TestLucene90NormsFormatMergeInstance, 
TestLucene90StoredFieldsFormatHighCompression, 
TestCompressingTermVectorsFormat, TestPerFieldPostingsFormat, 
TestDoubleRange, TestFieldType, TestLatLonDocValuesField, 
TestLatLonMultiLineShapeQueries, TestLatLonPointDistanceFeatureQuery, 
TestLatLonPolygonShapeDVQueries, TestLongHashSet, 
TestSortedSetDocValuesSetQuery, TestXYMultiPolygonShapeQueries, 
TestXYShape, TestGeoUtils, TestPolygon2D, TestXYLine, Test2BDocs, 
Test2BPostingsBytes, TestAddIndexes, TestAllFilesHaveCodecHeader, 
TestBinaryDocValuesUpdates, TestCodecHoldsOpenFiles, TestCrash, 
TestDemoParallelLeafReader, TestDocIDMerger, TestDocsAndPositions, 
TestDocumentsWriterStallControl, TestFieldInfos, TestFilterCodecReader, 
TestFlex, TestIndexCommit, TestIndexReaderClose, TestIndexWriterConfig, 
TestIndexWriterFromReader, TestIndexWriterNRTIsCurrent, 
TestIndexWriterReader, TestIndexingSequenceNumbers, TestLazyProxSkipping, 
TestMaxTermFrequency, TestMultiDocValues, TestNRTReaderWithThreads, 
TestNoMergePolicy, TestOmitPositions, 

Re: Help to find the RC of incompatible analyers

[Chris Hostetter]

You provided a list of TokenFilters that you use in your Analyzer, 
but you didn't mention anything about what Tokenizer you are using.

You also mentioned seeing a difference in the "tokenization result" and 
the example output you gave does in fact seem to be the output of the 
tokenizer -- not the output of the TokenFilters you mentioned -- since 
ShingleFilter would be producing more output tokens then you listed.

All of which suggests that the discrepency you are seeing is in your 
tokenizer.

Generally speaking: the best way to ensure folks on the mailing list can 
make sense of your situation and offer assistance is if you can provide 
reproducible snippets of code w/hardcoded input (ala unit tests) that 
demonstrates what you're seeing.

: Our current code is based on Lucene7.
: In some analyzer testcase, give a string "*Google's biologist’s*", the
: tokenization result is, *["google", "biologist"]*
: 
: But after I migrating the codebase to Lucene9,
: the result becomes, *["googles", "**biologist’s**"]*


: The analyzer uses the following three Lucene libraries:
: 
: org.apache.lucene.analysis.core.FlattenGraphFilter;
: 
: org.apache.lucene.analysis.shingle.ShingleFilter;
: 
: org.apache.lucene.analysis.synonym.SynonymGraphFilter;


-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: New branch and feature freeze for Lucene/Solr 8.10.0

[Chris Hostetter]

: I was referring to doing this with languages other than java.
: 
: I'm also assuming that exceeding this limit is going to cause indirect
: hassles for users of lucene, e.g. breaking various security / supply
: chain tools. We know a lot of these are total crap but people in the
: corporate world have to suffer under them.

Just to be clear -- our 'Implementation-Version:' has been exceeding the 
72 byte "single line" limit for a LONG time -- worrying about how 
"security / supply chain" tools will handle parsing that line now seems 
kind of silly...

If tools can't handle a line wrap in the 8.10 jars, then they haven't 
been able to handle the line wrap since we switched from svn to git (when
the Implementation Version values switched from being based svn version# 
to git sha)

The *ONLY* thing that's new here is where in the value the line wrap 
happens (with 8.10.0 it happens in the middle of the SHA) and that our 
smoketest tool isn't smart enough to parse the values properly.

This is not even the first time we've even had a conversation about the 
smoke tester and Implementation Version line wraps: LUCENE-7023.


: Its super-easy to use a short hash here and avoid problems.


There is however an actual and practical downside to switching our 
implementation version to using a "short" SHA, and that's that we would 
lose the ability to garuntee that the information in the 
Implementation-Version uniquely identifies what commit a given jar was 
built from.  Multiple commits with the same short(end) hash are possible 
-- Multiple commits with identical (full) commits is not.

Folks may think that using git tags is useful enough for figuring this 
out from official releases, but being able to look at the jar metadata 
from arbitrary builds off of arbitrary branches and sanity check where 
exactly they come from has been very useful to me for 10+ years.


: On Thu, Sep 16, 2021 at 3:03 AM Dawid Weiss  wrote:
: >
: > Jar command doesn't have it, true. But it's fairly trivial to do, even
: > with an inline snippet like this one?
: >
: > public class PrintManifest {
: >   public static void main(String[] jars) throws IOException {
: > for (var jar : jars) {
: >   var manifest = new JarFile(Paths.get(jar).toFile()).getManifest();
: >   var attrs = manifest.getMainAttributes();
: >   System.out.println(jar + ": " + 
attrs.getValue("Implementation-Version"));
: > }
: >   }
: > }
: >
: > I have this in my lucene-core-9.0.0-SNAPSHOT.jar:
: >
: > Implementation-Version: 9.0.0-SNAPSHOT de45b68c909815ce5ea7b6b9e1a2ce337
: >  5b6334d [snapshot build, details omitted]
: >
: > and running:
: >
: > java PrintManifest.java lucene-core-9.0.0-SNAPSHOT.jar
: >
: > shows:
: >
: > lucene-core-9.0.0-SNAPSHOT.jar: 9.0.0-SNAPSHOT
: > de45b68c909815ce5ea7b6b9e1a2ce3375b6334d [snapshot build, details
: > omitted]
: >
: > This seems easier to me than trying to remember and keep the length of
: > that line shorter than an arbitrary limit.
: >
: > Dawid
: >
: >
: > On Wed, Sep 15, 2021 at 9:46 PM Robert Muir  wrote:
: > >
: > > But its irrelevant that is "valid" when virtually no tools match it.
: > >
: > > In other words, I'd agree with you if the "jar" command had some
: > > ability to read these manifests and print stuff to stdout, e.g. if
: > > there was ANY interop at all here.
: > >
: > > But there isn't. So IMO it makes no sense to cause confusion and chaos
: > > by adding an unnecessarily long git commit hash.
: > >
: > > On Wed, Sep 15, 2021 at 3:26 PM Dawid Weiss  wrote:
: > > >
: > > >
: > > > This is valid manifest line-breaking though... Can we read the manifest 
properly on the smoke tester side somehow (for example, run a Java process that 
reads and extracts the required attribute)? This way we wouldn't care about the 
implementation details of how manifest wraps the lines (or escapes characters).
: > > >
: > > > D.
: > > >
: > > > On Wed, Sep 15, 2021 at 8:46 PM Mike Drob  wrote:
: > > >>
: > > >> The benchmark jar has the info we need… sort of. When I built it, it 
has:
: > > >>
: > > >> Implementation-Version: 8.10.0 75a5061d3715cc5d93c4cbe4f1fa62bf035eea1
: > > >>  1 - mdrob - 2021-09-15 11:40:36
: > > >>
: > > >>
: > > >> and it’s looking for Implementation-Version: 8.10.0 
75a5061d3715cc5d93c4cbe4f1fa62bf035eea11 on one line.
: > > >>
: > > >> Because 8.10 is a character longer than 8.9, we happen to wrap the 
last character of the git commit sha. From the manifest spec:
: > > >>
: > > >> No line may be longer than 72 bytes (not characters), in its 
UTF8-encoded form.
: > > >> If a value would make the initial line longer than this, it should be 
continued
: > > >> on extra lines (each starting with a single SPACE).
: > > >>
: > > >> And we were already teetering on the edge of that limit. We'll run 
into this problem again in a few years when we try to release version 10.0.0, 
so solving it now has practical benefits down the line.
: > > >>
: > > >> There's a few options that I can 

Re: [JENKINS] Lucene » Lucene-NightlyTests-main - Build # 215 - Unstable!

[Chris Hostetter]

: I did have to tweak the "reproduce with:" line to reproduce it though...
: maybe because it was a timeout?
: 
: It said: --tests
: "org.apache.lucene.index.TestDocumentsWriterStallControl.classMethod"
: But it should be: --tests
: 
"org.apache.lucene.index.TestDocumentsWriterStallControl.testAcquireReleaseRace"
: ?
: Or maybe no test method at all since we don't know who is truly to blame?

There are two failures reported, w/two reproduce lines in the output.  One 
is testAcquireReleaseRace, the other is 'classMethod' -- IIUC classMethod 
is the generic way the gradle test runner reports that something failed at 
the suite (or static class init?) level ... like an AfterClass method, or 
as Dawid said: a suite timeout (which i think is inforced via a ClassRule? 
... not sure ...)

I think that in the case of a suite timeout you'll always see that test + 
'classMethod' failure pattern -- unless the test finishes in a timely 
manner and it's some AfterClass cleanup that "hangs"


-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene » Lucene-Solr-SmokeRelease-8.x - Build # 147 - Still Failing!

[Chris Hostetter]

Is this perhaps related to LUCENE-9836 ?


: Date: Fri, 19 Mar 2021 01:34:14 + (UTC)
: From: Apache Jenkins Server 
: To: hoss...@apache.org
: Subject: [JENKINS] Lucene » Lucene-Solr-SmokeRelease-8.x - Build # 147 - Still
:  Failing!
: 
: Build: 
https://ci-builds.apache.org/job/Lucene/job/Lucene-Solr-SmokeRelease-8.x/147/
: 
: No tests ran.
: 
: Build Log:
: [...truncated 29479 lines...]
: prepare-release-no-sign:
: [mkdir] Created dir: 
/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-SmokeRelease-8.x/lucene/build/smokeTestRelease/dist
:  [copy] Copying 487 files to 
/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-SmokeRelease-8.x/lucene/build/smokeTestRelease/dist/lucene
:  [copy] Copying 211 files to 
/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-SmokeRelease-8.x/lucene/build/smokeTestRelease/dist/solr
:[smoker] Java 1.8 JAVA_HOME=/home/jenkins/tools/java/latest1.8
:[smoker] NOTE: output encoding is UTF-8
:[smoker] 
:[smoker] Load release URL 
"file:/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-SmokeRelease-8.x/lucene/build/smokeTestRelease/dist/"...
:[smoker] 
:[smoker] Get KEYS...
:[smoker] Downloading online KEYS file 
https://archive.apache.org/dist/lucene/KEYS
:[smoker] Downloading https://archive.apache.org/dist/lucene/KEYS
:[smoker] 0.3 MB in 1.03 sec (0.3 MB/sec)
:[smoker] 
:[smoker] Test Lucene...
:[smoker]   test basics...
:[smoker]   check changes HTML...
:[smoker]   download lucene-8.9.0-src.tgz...
:[smoker] Downloading 
file:///home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-SmokeRelease-8.x/lucene/build/smokeTestRelease/dist/lucene/lucene-8.9.0-src.tgz
:[smoker] 47.2 MB in 0.04 sec (1282.9 MB/sec)
:[smoker] verify sha512 digest
:[smoker]   download lucene-8.9.0.tgz...
:[smoker] Downloading 
file:///home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-SmokeRelease-8.x/lucene/build/smokeTestRelease/dist/lucene/lucene-8.9.0.tgz
:[smoker] 81.0 MB in 0.07 sec (1237.1 MB/sec)
:[smoker] verify sha512 digest
:[smoker]   download lucene-8.9.0.zip...
:[smoker] Downloading 
file:///home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-SmokeRelease-8.x/lucene/build/smokeTestRelease/dist/lucene/lucene-8.9.0.zip
:[smoker] 93.3 MB in 0.08 sec (1189.2 MB/sec)
:[smoker] verify sha512 digest
:[smoker]   unpack lucene-8.9.0.tgz...
:[smoker] verify JAR metadata/identity/no javax.* or java.* classes...
:[smoker] test demo with 1.8...
:[smoker]   got 4313 hits for query "lucene"
:[smoker] checkindex with 1.8...
:[smoker] check Lucene's javadoc JAR
:[smoker]   unpack lucene-8.9.0.zip...
:[smoker] verify JAR metadata/identity/no javax.* or java.* classes...
:[smoker] test demo with 1.8...
:[smoker]   got 4313 hits for query "lucene"
:[smoker] checkindex with 1.8...
:[smoker] check Lucene's javadoc JAR
:[smoker]   unpack lucene-8.9.0-src.tgz...
:[smoker] make sure no JARs/WARs in src dist...
:[smoker] run "ant validate"
:[smoker] run tests w/ Java 8 and testArgs='-Dtests.badapples=false 
-Dtests.slow=false'...
:[smoker] test demo with 1.8...
:[smoker]   got 234 hits for query "lucene"
:[smoker] checkindex with 1.8...
:[smoker] generate javadocs w/ Java 8...
:[smoker] 
:[smoker] Crawl/parse...
:[smoker] 
:[smoker] Verify...
:[smoker]   confirm all releases have coverage in TestBackwardsCompatibility
:[smoker] find all past Lucene releases...
:[smoker] run TestBackwardsCompatibility..
:[smoker] success!
:[smoker] 
:[smoker] Test Solr...
:[smoker]   test basics...
:[smoker]   check changes HTML...
:[smoker]   download solr-8.9.0-src.tgz...
:[smoker] Downloading 
file:///home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-SmokeRelease-8.x/lucene/build/smokeTestRelease/dist/solr/solr-8.9.0-src.tgz
:[smoker] 129.6 MB in 1.45 sec (89.1 MB/sec)
:[smoker] verify sha512 digest
:[smoker]   download solr-8.9.0.tgz...
:[smoker] Downloading 
file:///home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-SmokeRelease-8.x/lucene/build/smokeTestRelease/dist/solr/solr-8.9.0.tgz
:[smoker] 187.5 MB in 1.09 sec (172.3 MB/sec)
:[smoker] verify sha512 digest
:[smoker]   download solr-8.9.0.zip...
:[smoker] Downloading 
file:///home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-SmokeRelease-8.x/lucene/build/smokeTestRelease/dist/solr/solr-8.9.0.zip
:[smoker] 188.5 MB in 1.17 sec (161.2 MB/sec)
:[smoker] verify sha512 digest
:[smoker]   unpack solr-8.9.0.tgz...
:[smoker] verify JAR metadata/identity/no javax.* or java.* classes...
:[smoker] unpack lucene-8.9.0.tgz...
:[smoker]   **WARNING**: skipping check of 

Re: Random disabling of asserts in tests is not working

[Chris Hostetter]

FWIW: I don't remember it ever being randomizable -- because it has to be 
set as a commandline arg on the JVM.  Users have always been able to set 
it to either true|false from the ant/gradle command line however (maybe 
Mike was thinking of jenkins jobs that randomize it? ... i thought uwe's 
jenkins box use to randomize it as part of the VM randomization but I 
don't think that's currently true.

-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-Solr-8.x-Linux (64bit/jdk-11.0.6) - Build # 5837 - Still Unstable!

[Chris Hostetter]

https://issues.apache.org/jira/browse/SOLR-15149#

: Date: Tue, 16 Feb 2021 17:55:17 + (UTC)
: From: Policeman Jenkins Server 
: Reply-To: dev@lucene.apache.org
: To: bui...@lucene.apache.org
: Subject: [JENKINS] Lucene-Solr-8.x-Linux (64bit/jdk-11.0.6) - Build # 5837 -
: Still Unstable!
: 
: Build: https://jenkins.thetaphi.de/job/Lucene-Solr-8.x-Linux/5837/
: Java: 64bit/jdk-11.0.6 -XX:+UseCompressedOops -XX:+UseSerialGC
: 
: 24 tests failed.
: FAILED:  
org.apache.solr.ltr.model.TestLinearModel.notExistentStore_shouldThrowMeaningFulException
: 
: Error Message:
: expected:<...del.ModelException: [Feature Store not found]: 
not_existent_store> but was:<...del.ModelException: [Missing or empty feature 
store]: not_existent_store>
: 
: Stack Trace:
: org.junit.ComparisonFailure: expected:<...del.ModelException: [Feature Store 
not found]: not_existent_store> but was:<...del.ModelException: [Missing or 
empty feature store]: not_existent_store>
:   at 
__randomizedtesting.SeedInfo.seed([3E470D0E12452BE8:8A5D11F9D9B486F8]:0)
:   at org.junit.Assert.assertEquals(Assert.java:117)
:   at org.junit.Assert.assertEquals(Assert.java:146)
:   at 
org.apache.solr.ltr.model.TestLinearModel.notExistentStore_shouldThrowMeaningFulException(TestLinearModel.java:241)
:   at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
:   at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
:   at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
:   at java.base/java.lang.reflect.Method.invoke(Method.java:566)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
:   at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
:   at org.junit.rules.RunRules.evaluate(RunRules.java:20)
:   at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
:   at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
:   at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
:   at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
:   at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
:   at org.junit.rules.RunRules.evaluate(RunRules.java:20)
:   at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
:   at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
:   at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
:   at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
:   at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
:   at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
:   at org.junit.rules.RunRules.evaluate(RunRules.java:20)
:   at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
:   at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
:   at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
:   at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
:   at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
:   at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
:   at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
:   at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
:   at 

Re: package-tgz-src / package-src-tgz equivilent missing from gradle build on master?

[Chris Hostetter]

: At this major juncture in the project (switching build systems), it's worth
: re-considering what we need to produce on a release.  A -src.tar.gz seems
: anachronistic to me given both (a) GitHub source control providing a
: convenient download .tar.gz for any release "tag" --
: https://github.com/apache/lucene-solr/releases, and (b) maven central
: having -source.jar for convenience in IDE tooling.  So why bother?

Because we're required to by ASF policy...

http://www.apache.org/legal/release-policy.html#artifacts

"Every ASF release MUST contain one or more source packages, which MUST be 
sufficient for a user to build and test the release provided they have 
access to the appropriate platform and tools."

...

"As a convenience to users that might not have the appropriate tools to 
build a compiled version of the source, binary/bytecode packages MAY be 
distributed alongside official Apache releases. In all such cases, the 
binary/bytecode package MUST have the same version number as the source 
release and MUST only add binary/bytecode files that are the result of 
compiling that version of the source code release and its dependencies."


-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene » Lucene-Solr-Tests-8.x - Build # 1525 - Still Failing!

[Chris Hostetter]

This appears to have been broken by fcb922bea18fde00d0d657947b1554addb49b47e ...

The javadocs were changed from  to  confusing 
checkJavadocLinks.py and breaking precommit...

(I would have re-opend the corrisponding jira, but aparently this change 
was made w/o a jira to track it?)




: Date: Thu, 4 Feb 2021 21:21:26 + (UTC)
: From: Apache Jenkins Server 
: Reply-To: dev@lucene.apache.org
: To: bui...@lucene.apache.org
: Subject: [JENKINS] Lucene » Lucene-Solr-Tests-8.x - Build # 1525 - Still
: Failing!
: 
: Build: https://ci-builds.apache.org/job/Lucene/job/Lucene-Solr-Tests-8.x/1525/
: 
: All tests passed
: 
: Build Log:
: [...truncated 57454 lines...]
: -documentation-lint:
:  [echo] checking for broken html...
: [jtidy] Checking for broken html (such as invalid tags)...
:[delete] Deleting directory 
/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-Tests-8.x/lucene/build/jtidy_tmp
:  [echo] Checking for broken links...
:  [exec] 
:  [exec] Crawl/parse...
:  [exec] 
:  [exec]   
file:///build/docs/backward-codecs/org/apache/lucene/codecs/lucene70/Lucene70NormsFormat.html
:  [exec]   WARNING: failed to parse 
file:///build/docs/backward-codecs/org/apache/lucene/codecs/lucene70/Lucene70NormsFormat.html:
:  [exec] Traceback (most recent call last):
:  [exec]   File 
"/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-Tests-8.x/dev-tools/scripts/checkJavadocLinks.py",
 line 110, in parse
:  [exec] parser.feed(html)
:  [exec]   File "/usr/lib/python3.6/html/parser.py", line 111, in feed
:  [exec] self.goahead(0)
:  [exec]   File "/usr/lib/python3.6/html/parser.py", line 171, in goahead
:  [exec] k = self.parse_starttag(i)
:  [exec]   File "/usr/lib/python3.6/html/parser.py", line 345, in 
parse_starttag
:  [exec] self.handle_starttag(tag, attrs)
:  [exec]   File 
"/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-Tests-8.x/dev-tools/scripts/checkJavadocLinks.py",
 line 80, in handle_starttag
:  [exec] raise RuntimeError('couldn\'t find an href nor name in link 
in %s: only got these attrs: %s' % (self.baseURL, attrs))
:  [exec] RuntimeError: couldn't find an href nor name in link in 
file:///build/docs/backward-codecs/org/apache/lucene/codecs/lucene70/Lucene70NormsFormat.html:
 only got these attrs: [('id', 'nvm')]
:  [exec] 
:  [exec] Verify...
:  [exec] 
:  [exec] Broken javadocs links were found! Common root causes:
:  [exec] * A typo of some sort for manually created links.
:  [exec] * Public methods referencing non-public classes in their 
signature.
: 
: BUILD FAILED
: 
/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-Tests-8.x/build.xml:634:
 The following error occurred while executing this line:
: 
/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-Tests-8.x/build.xml:101:
 The following error occurred while executing this line:
: 
/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-Tests-8.x/lucene/build.xml:142:
 The following error occurred while executing this line:
: 
/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-Tests-8.x/lucene/build.xml:155:
 The following error occurred while executing this line:
: 
/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-Tests-8.x/lucene/common-build.xml:2552:
 exec returned: 1
: 
: Total time: 67 minutes 8 seconds
: Build step 'Invoke Ant' marked build as failure
: Archiving artifacts
: Recording test results
: Email was triggered for: Failure - Any
: Sending email for trigger: Failure - Any
: 

-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: package-tgz-src / package-src-tgz equivilent missing from gradle build on master?

[Chris Hostetter]

: It wasn't as much of an oversight as lack of knowledge how to deal
: with it. I personally think the source distribution should be
: equivalent to a clean git checkout. If this works for you then things
: are relatively simple and I can do it.

I'm not an authority here, so what works for me doesn't really matter :)

While i would generally agree with you, I think there are some aspects of 
how the current targets work that are worth bearing in mind since 
deliberate decisions were made to get to this point - deliberate 
decisions should probably made to "undo" these choices..

* excluding jdk javadoc package-list files (for licensing reasons evidently)
* building Changes.html from CHANGES.txt
* including lucnee/CHANGES.txt in solr as LUCENE_CHANGES.txt
* setting chmod bits on scripts


: On Wed, Feb 3, 2021 at 1:30 AM Chris Hostetter  
wrote:
: >
: >
: > thinking about how we (want to) build solr docker containers moving
: > forward (SOLR-15102) lead me to realize that on the mster branch, there
: > doesn't seem to be any logic to build the "*-src.tgz" files.
: >
: > On branch_8x "ant package" in both the lucene & solr directories handles
: > this by delegation to either package-tgz-src or package-src-tgz (for some
: > reason they have diff names in each dir?) ... but there doesn't seem to be
: > any equivilent of this logic in the gradle build.
: >
: > Was this an oversight, or is ther some other plan for where/how we deal
: > with "src.tgz" release artifacts in 9.x ?
: >
: >
: > -Hoss
: > http://www.lucidworks.com/
: >
: > -
: > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
: > For additional commands, e-mail: dev-h...@lucene.apache.org
: >
: 
: -
: To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
: For additional commands, e-mail: dev-h...@lucene.apache.org
: 
: 

-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

package-tgz-src / package-src-tgz equivilent missing from gradle build on master?

[Chris Hostetter]

thinking about how we (want to) build solr docker containers moving 
forward (SOLR-15102) lead me to realize that on the mster branch, there 
doesn't seem to be any logic to build the "*-src.tgz" files.


On branch_8x "ant package" in both the lucene & solr directories handles 
this by delegation to either package-tgz-src or package-src-tgz (for some 
reason they have diff names in each dir?) ... but there doesn't seem to be 
any equivilent of this logic in the gradle build.


Was this an oversight, or is ther some other plan for where/how we deal 
with "src.tgz" release artifacts in 9.x ?



-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Solr Docker discussion

[Chris Hostetter]

FYI: I've implemented a PoC of this idea and opened SOLR-15127...

https://issues.apache.org/jira/browse/SOLR-15127


: Date: Fri, 22 Jan 2021 14:37:19 -0700 (MST)
: From: Chris Hostetter 
: To: Solr/Lucene Dev 
: Subject: Re: Solr Docker discussion
: 
: 
: : So I've taken a look at the Elastic docker images [1], and they are
: : structured much like official images, except for the use of multi-stage
: : builds. They have no ARGs, and the docker context is merely the directory
: : in which the Dockerfile sits. With this in mind, and wanting to still take
: : advantage of the Official Docker Image, we will likely need to do some sort
: : of templating, as Hoss suggested. This can be automated using gradle fairly
: : easily I would imagine, and we could ensure that the only changes it makes
: : is to the part of the image that downloads Solr. That way there is very
: : little room for differences to occur between local and release images.
: 
: If docker-library is willing to allow _/solr:X.Y images that are just 
: simple 'FROM apache/solr:X.Y' wrappers, and the apache/solr iamges can be 
: built with multi-stage builds, then i'm not sure we would really need 
: templating of the Dockerfile -- or to download a solr.tgz file.
: 
: The Dockerfile could have an initial stage that runs "gradle dist" to 
: compile the java source files -- using a build AEG to override the 
: default X.Y.Z-SNAPSHOT version variables -- while a later stage could 
: then take the resulting solr.tgz from the first stage and setup the file 
: layout for the final image.
: 
: So users wanting to build a "snapshot" docker image from their local 
: (possibly patched) checkout could just do...
: 
:   docker build -f solr/docker/Dockerfile .
: 
: Or from a remote git branch:
: 
:   docker build -f solr/docker/Dockerfile 
https://gitbox.apache.org/repos/asf/lucene-solr.git#branch_9x
: 
: while the official apache/solr:X.Y.Z image would could be reproducibly 
: buildable via...
: 
:   docker build --build-arg SOLR_VERSION=X.Y.Z -f solr/docker/Dockerfile 
https://gitbox.apache.org/repos/asf/lucene-solr.git#releases/lucene-solr/X.Y.Z
: 
: Same Dockerfile in all use cases, no need for intermediate downloading 
: from any remote servers.
: 
: 
: -Hoss
: http://www.lucidworks.com/
: 

-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Solr Docker discussion

[Chris Hostetter]

: So I've taken a look at the Elastic docker images [1], and they are
: structured much like official images, except for the use of multi-stage
: builds. They have no ARGs, and the docker context is merely the directory
: in which the Dockerfile sits. With this in mind, and wanting to still take
: advantage of the Official Docker Image, we will likely need to do some sort
: of templating, as Hoss suggested. This can be automated using gradle fairly
: easily I would imagine, and we could ensure that the only changes it makes
: is to the part of the image that downloads Solr. That way there is very
: little room for differences to occur between local and release images.

If docker-library is willing to allow _/solr:X.Y images that are just 
simple 'FROM apache/solr:X.Y' wrappers, and the apache/solr iamges can be 
built with multi-stage builds, then i'm not sure we would really need 
templating of the Dockerfile -- or to download a solr.tgz file.

The Dockerfile could have an initial stage that runs "gradle dist" to 
compile the java source files -- using a build AEG to override the 
default X.Y.Z-SNAPSHOT version variables -- while a later stage could 
then take the resulting solr.tgz from the first stage and setup the file 
layout for the final image.

So users wanting to build a "snapshot" docker image from their local 
(possibly patched) checkout could just do...

  docker build -f solr/docker/Dockerfile .

Or from a remote git branch:

  docker build -f solr/docker/Dockerfile 
https://gitbox.apache.org/repos/asf/lucene-solr.git#branch_9x

while the official apache/solr:X.Y.Z image would could be reproducibly 
buildable via...

  docker build --build-arg SOLR_VERSION=X.Y.Z -f solr/docker/Dockerfile 
https://gitbox.apache.org/repos/asf/lucene-solr.git#releases/lucene-solr/X.Y.Z

Same Dockerfile in all use cases, no need for intermediate downloading 
from any remote servers.


-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene » Lucene-Solr-Check-master - Build # 1444 - Still Unstable!

[Chris Hostetter]

pretty sure this is https://issues.apache.org/jira/browse/SOLR-14067

: Date: Fri, 22 Jan 2021 20:20:43 + (UTC)
: From: Apache Jenkins Server 
: Reply-To: dev@lucene.apache.org
: To: bui...@lucene.apache.org
: Subject: [JENKINS] Lucene » Lucene-Solr-Check-master - Build # 1444 - Still
: Unstable!
: 
: Build: 
https://ci-builds.apache.org/job/Lucene/job/Lucene-Solr-Check-master/1444/
: 
: 454 tests failed.
: FAILED:  org.apache.solr.cluster.events.ClusterEventProducerTest.testEvents
: 
: Error Message:
: java.lang.AssertionError: Timed out waiting for expected event NODES_DOWN
: 
: Stack Trace:
: java.lang.AssertionError: Timed out waiting for expected event NODES_DOWN
:   at 
__randomizedtesting.SeedInfo.seed([7CD9B2A849453506:BFBE3B166D1D9756]:0)
:   at org.junit.Assert.fail(Assert.java:89)
:   at 
org.apache.solr.cluster.events.AllEventsListener.waitForExpectedEvent(AllEventsListener.java:54)
:   at 
org.apache.solr.cluster.events.ClusterEventProducerTest.testEvents(ClusterEventProducerTest.java:125)
:   at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
:   at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
:   at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
:   at java.base/java.lang.reflect.Method.invoke(Method.java:566)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:942)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:978)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
:   at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
:   at org.junit.rules.RunRules.evaluate(RunRules.java:20)
:   at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:44)
:   at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
:   at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
:   at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
:   at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
:   at org.junit.rules.RunRules.evaluate(RunRules.java:20)
:   at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
:   at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
:   at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:819)
:   at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:470)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:951)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:836)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:887)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
:   at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
:   at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
:   at org.junit.rules.RunRules.evaluate(RunRules.java:20)
:   at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
:   at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
:   at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
:   at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
:   at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
:   at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
:   at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
:   at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:51)
:   at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
:   at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
:   at 

Re: Solr Docker discussion

[Chris Hostetter]

: I'd recommend building apache/solr:x.y.z as part of the normal solr 
: release, then import those images into the official images library so 
...
: I don't know exactly what it takes for this to happen on the 
: official-image library side, but it seems that Elastisearch does this 

Oh wow ... ok, yeah: I had no idea that was allowed by docker-library 
based on the things I'd read about "official" docker images.  (per the 
links in the comments, the Dockerfile for the ES base image even uses 
multi-stage builds which would help seriously simplify the our 'official' 
image build process)

Yeah, if this is something docker-library will allow us to do as well -- 
define _/solr images directly 'FROM' apache/solr images -- and that 
those are/can be reproducibly built from a (multistage) Dockerfile in our 
git source repo identified by our release tag -- then by all means that 
seems like the best way to go.   

(I suspect the one finicky bit may be ensuring that docker-library is 
ok with us using '--build-arg' to set the version when building our base
apache/solr images so that by building docker images from source checkout 
defaults to building a snapshot)


-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Solr Docker discussion

[Chris Hostetter]

:- _/solr - docker run solr:9.0 (Official Docker Image)
:- apache/solr - docker run apache/solr:9.0
: 
: The benefits of the first are 1) the nice usability of being able to
: plainly specify "solr" and 2) the "Docker Official Images" badge on
: DockerHub. The downsides are that there are very strict requirements with
: creating Official Docker Images, which would complicate and require
: separating the way that we build release docker images and local docker
: images.

I don't know that this is strictly true?  I think it would be possible to 
support "local" images and "release" images fairly easily w/ a small 
amount of tooling, but more on this idea farther below...


: *I think there is some consensus that choosing the "apache/solr" location
: is fine, and worth the added flexibility we get in the build process.*
...
: *The current opinion in the community is to abandon the "Docker Official
: Images" style process of downloading and validating official binaries, and
: instead having the release manager use the local-build image creation with
: the final release source.* This should result in the same docker image in
: the end, however there is no trust built into the docker image itself.
: Instead we are likely going to document a way for users to verify the
: docker-image contents themselves.

"Citation Needed" :) 

Seriously though ... can you point me to when/where there has been 
discussion/concensus about this?  

And which community exactly are you refering to? -- Because I suspect that 
on the subject of where/how to build the Docker images and if/how-much it 
matters wether they live in _/solr vs apache/solr you may find a 
discrepency between the "community of solr comitters" and the "docker user 
community".  Keep in mind: Mak didn't jump through the neccessary hoops 
all those years ago to create _/solr just because he thought it would be 
fun -- it would have been a lot easier to just create & publish 
makuk66/solr images, but there was a (docker) community desire to have 
"Docker official" _/solr images.

I think it would be presumptuous to assume that (docker) community desire 
is going to go away just because the (apache solr) community is now 
willing to publish apache/solr images.


With that in mind, I *think* there is a relatively straight forward way we 
can have a "best of both worlds" type situation

DISCLAIMER: I am not a docker expect, nor am i an expert on the docker-hub 
rules for "official" images -- the idea below is just based on how the 
current docker-solr/docker-solr github repo works. (I should also mention 
I'm not up to speed on the docker related bits in lucene-solr master for 
building "local" images, for now I'm just ignoring that and assuming it's 
fair game to start over from scratch with how building docker images from 
a source release/checkout work)

The crux of my suggestion is that we keep a Dockerfile.template in 
the lucene-solr repo, on master (and any future branches) which has 
templated values for the SOLR_VERSION and SHA512 of the 
"solr-SOLR_VERSION.tgz" file names.

as part of the "gradle assemble" process, right after gradle creates 
./packaging/build/distributions/solr-FOO.tgz and 
./packaging/build/distributions/solr-FOO.tgz.sha512, the 
Dockerfile.template should be used to create Dockerfile with the 
corisponding file names and SHA512 values hardcoded into it.

the Dockerfile(.template) should have COPY directive that looks something 
like...

  COPY ./solr-FOO.tgz* /opt/solr/dist-archive

...followed by a RUN command invoking bash conditionals to download 
solr-FOO.tgz from SOLR_DOWNLOAD_URL into /opt/solr/dist-archive if and 
only if /opt/solr/dist-archive/solr-FOO.tgz doesn't already exist -- and 
if it does exist to touch some file 
/opt/solr/dist-archive/this.is.a.local.build.txt

RUN script logic after this should attempt GPG validation (via KEYS & asc 
files fetched from dist.apache.org) if and only if 
/opt/solr/dist-archive/this.is.a.local.build.txt does not exist. (Either 
way the hardcoded SHA512 value should be used to validate the tgz file 
matches expectations)


This should allow us to have "Local" docker builds that don't require 
any network access or downloading special files either by putting the 
(post-templating) Dockerfile directly in ./packaging/build/distributions/ 
and telling people to use that as the context for "docker build" or by 
some other gradle helper logic for constructing a special "docker context 
dir" for people to build from where gradle copies the solr-FOO.tgz file.


"Official" docker-solr builds could use the exact same Dockerfile -- 
committed to the docker-solr/docker-solr (or some future 
apache/docker-solr) repo by the RM via some release publishing scripts

(Any other "files" in the lucene-solr git repo that we want to be part 
of docker images could likewise be copied to the "local" docker context 
dir by gradle and/or to the apache/docker-solr repo by the release 

Re: Separate git repo(s) for Solr modules

[Chris Hostetter]

: As we discussed over the last few months, there seems a need to move
: non-core pieces away from the Solr core module. The contribs are presently
: a good place, but it makes sense to have a separate git repository hosting
: such modules. Some candidates that come to mind are the present day contrib

can you explain why it makes sense to have a separate git repo for these 
things?

I can think of lots of reasons why it makes sense to have a single 
repo for all things solr (unified CI that quickly identifies if core 
changes break "first order" plugins, shared feature branches & monotomic
commits of code that affects APIs and impls of those APIs, etc...) but 
I've yet to see any concrete specifics of why multiple git repos are 
"better" then just having distinct sub-projects (with distinct artifacts) 
in the same repo other then "it makes sense"

why does it make sense?

why can't the ideas of "solr-sandbox" and "solr-extras" just be 
directories in the "solr repo" ? ... what value is gained by making them 
new repos?


-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Solr CHANGES.txt 8.8 adjustment proposal

[Chris Hostetter]

: Also, I see an "Upgrade Notes" section, that Hossman added (CC'ed), but I
: think as a community we agreed to omit such things from our CHANGES.txt and
: instead rely on solr-upgrade-notes.adoc (referenced at the *very* top of

I don't remember that decision being made -- but ok, fair enough...

If the plan moving forward is that there should be a pre-determined set of 
headers, and that devs shouldn't be adding headings like "Upgrade Notes" 
as needed anymore, then I would like to suggest someone 
should probably make 'changes2html.pl' (or some othe tool run on 
'check') fail if an unexpected header is added?



-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Poor tool detail on lucene.apache.org/solr/ site

[Chris Hostetter]

: The detail on the lucene.apache.org/solr/ is very brief and poor.  It 
: seems you are just pushing folks to buy the newest SOLR book.  very sad, 
: who new is going to use this tool if the documentation detail is lousy.  
: Most will just go buy a well document non-open sorce tool that is well 
: supported.

So there is a "Books" section on lucene.apache.org/solr/ -- but it's 
basically the last thing on the page, after the sections on "Learn more 
about Solr" (which inlcudes links to the "Resources" page where all the 
free documentation are -- also promomently featured in the main nav, in 
red, at the top of the page) and and aftet the "Getting Started" section 
which includes a direct link to "Read the Tutorial" and another "Level Up" 
call linking again to the detailed "More Resources" (which is all free 
documentation)

So from my perspective it's confusing that your perspetive of the page is 
that the "focus" is on selling books?

At the same time: your perspective as a new user and your "first 
impression" of the website is fascinating to me -- and I geniuniely 
encourage you to elaborate more on how you think the main web page should 
be structured to better draw attention to the important content for new 
users.

i say with some intentional irony: as a person railing against the solr 
website for being too brief, your email was very brief on details about 
what exactly you suggest we change.  This is an open source project, even 
if you aren't a java developer and/or have a detailed understanding of the 
code, your advice/contributions to the 'new user' experience can be a 
valuable way to contribute.  If you want to see changes made, be specific 
about what changes you suggest: 
On topic: 
https://www.youtube.com/watch?v=wxL3trl6-fI
https://people.apache.org/~hossman/activate2018/



-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Failing test on branch_8x: org.apache.solr.rest.schema.TestBulkSchemaAPI.testCopyFieldWithReplace

[Chris Hostetter]

: WHen I try your exact repro line (along with the method name), I get:
: BUILD FAILED
: /home/ishan/code/lucene-solr/lucene/common-build.xml:1616: Not even a
: single test was executed (a typo in the filter pattern maybe?).

your repo is stale Ishan -- it's a test that was committed today...

https://issues.apache.org/jira/browse/SOLR-14950


: On Thu, Jan 7, 2021 at 11:09 PM Timothy Potter 
: wrote:
: 
: > This test seems to fail consistently for me on 8x:
: >
: >
: > org.apache.solr.rest.schema.TestBulkSchemaAPI.testCopyFieldWithReplace
: >
: >
: > NOTE: reproduce with: ant test  -Dtestcase=TestBulkSchemaAPI
: > -Dtests.method=testCopyFieldWithReplace -Dtests.seed=7657D4FA16A76DC8
: > -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=es-UY
: > -Dtests.timezone=America/Resolute -Dtests.asserts=true
: > -Dtests.file.encoding=UTF-8
: >
: > java.lang.AssertionError: {
: >   "responseHeader":{
: > "status":400,
: > "QTime":6},
: >   "error":{
: > "metadata":[
: >   "error-class","org.apache.solr.api.ApiBag$ExceptionWithErrObject",
: >
: > "root-error-class","org.apache.solr.api.ApiBag$ExceptionWithErrObject"],
: > "details":[{
: > "add-field-type":{
: >   "name":"myNewTextField",
: >   "class":"solr.TextField",
: >   "analyzer":{
: > "charFilters":[{
: > "name":"patternReplace",
: > "replacement":"$1$1",
: > "pattern":"([a-zA-Z])1+"}],
: > "tokenizer":{"name":"whitespace"},
: > "filters":[{"name":"asciiFolding"}]}},
: > "errorMessages":["Every charFilter must define a class
: > property!\n"]}],
: > "msg":"error processing commands",
: > "code":400}}
: >  expected null, but was:<{metadata=[error-class,
: > org.apache.solr.api.ApiBag$ExceptionWithErrObject, root-error-class,
: > org.apache.solr.api.ApiBag$ExceptionWithErrObject],
: > details=[{add-field-type={name=myNewTextField, class=solr.TextField,
: > analyzer={charFilters=[{name=patternReplace, replacement=$1$1,
: > pattern=([a-zA-Z])\\1+}], tokenizer={name=whitespace},
: > filters=[{name=asciiFolding}]}}, errorMessages=[Every charFilter must
: > define a class property!
: > ]}], msg=error processing commands, code=400}>
: >
: > at __randomizedtesting.SeedInfo.seed([7657D4FA16A76DC8:B096864C3F5D3F33]:0)
: > at org.junit.Assert.fail(Assert.java:89)
: > at org.junit.Assert.failNotNull(Assert.java:756)
: > at org.junit.Assert.assertNull(Assert.java:738)
: > at
: > 
org.apache.solr.rest.schema.TestBulkSchemaAPI.testCopyFieldWithReplace(TestBulkSchemaAPI.java:734)
: > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
: > at
: > 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
: > at
: > 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
: > at java.lang.reflect.Method.invoke(Method.java:498)
: >
: 

-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene » Lucene-Solr-Check-master - Build # 972 - Unstable!

[Chris Hostetter]

https://issues.apache.org/jira/browse/SOLR-15026

: Date: Wed, 2 Dec 2020 18:47:51 + (UTC)
: From: Apache Jenkins Server 
: Reply-To: dev@lucene.apache.org
: To: bui...@lucene.apache.org
: Subject: [JENKINS] Lucene » Lucene-Solr-Check-master - Build # 972 - Unstable!
: 
: Build: 
https://ci-builds.apache.org/job/Lucene/job/Lucene-Solr-Check-master/972/
: 
: 1 tests failed.
: FAILED:  
org.apache.solr.cloud.MiniSolrCloudClusterTest.testSolrHomeAndResourceLoaders
: 
: Error Message:
: org.apache.solr.client.solrj.SolrServerException: IOException occurred when 
talking to server at: https://127.0.0.1:34219/solr
: 
: Stack Trace:
: org.apache.solr.client.solrj.SolrServerException: IOException occurred when 
talking to server at: https://127.0.0.1:34219/solr
:   at 
__randomizedtesting.SeedInfo.seed([806A85748BD81F48:37548FA7602CB5FD]:0)
:   at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:712)
:   at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:269)
:   at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
:   at 
org.apache.solr.client.solrj.impl.LBSolrClient.doRequest(LBSolrClient.java:390)
:   at 
org.apache.solr.client.solrj.impl.LBSolrClient.request(LBSolrClient.java:360)
:   at 
org.apache.solr.client.solrj.impl.BaseCloudSolrClient.sendRequest(BaseCloudSolrClient.java:1168)
:   at 
org.apache.solr.client.solrj.impl.BaseCloudSolrClient.requestWithRetryOnStaleState(BaseCloudSolrClient.java:931)
:   at 
org.apache.solr.client.solrj.impl.BaseCloudSolrClient.request(BaseCloudSolrClient.java:865)
:   at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:229)
:   at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:246)
:   at 
org.apache.solr.cloud.MiniSolrCloudClusterTest.testSolrHomeAndResourceLoaders(MiniSolrCloudClusterTest.java:125)
:   at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
:   at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
:   at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
:   at java.base/java.lang.reflect.Method.invoke(Method.java:566)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:942)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:978)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
:   at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
:   at org.junit.rules.RunRules.evaluate(RunRules.java:20)
:   at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
:   at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
:   at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
:   at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
:   at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
:   at org.junit.rules.RunRules.evaluate(RunRules.java:20)
:   at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
:   at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
:   at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:819)
:   at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:470)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:951)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:836)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:887)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
:   at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
:   at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
:   at org.junit.rules.RunRules.evaluate(RunRules.java:20)
:   at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
:   at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
:   at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
:   at 

Re: [JENKINS] Lucene » Lucene-Solr-Check-master - Build # 698 - Failure!

[Chris Hostetter]

The failure history shows it really spiked up sometime just before Nov 
2nd.  If folks know of sees that reliably reproduce, who wants to run git 
bisect to try and track down the instigating commit?

http://fucit.org/solr-jenkins-reports/history-trend-of-recent-failures.html#series/org.apache.solr.TestRandomDVFaceting.testRandomFaceting


: Date: Wed, 4 Nov 2020 18:05:27 +0100
: From: Andrzej Białecki 
: Reply-To: dev@lucene.apache.org
: To: dev@lucene.apache.org
: Subject: Re: [JENKINS] Lucene » Lucene-Solr-Check-master - Build # 698 -
: Failure!
: 
: This is frequently failing for me too, seed 4B403C84E8713580 seems to always 
reproduce it on my local machine.
: 
: > On 3 Nov 2020, at 14:14, Dawid Weiss  wrote:
: > 
: > These randomized test failures reproduce for me in this test.
: > 
: > Dawid
: > 
: > On Tue, Nov 3, 2020 at 8:30 AM Apache Jenkins Server
: >  wrote:
: >> 
: >> Build: 
https://ci-builds.apache.org/job/Lucene/job/Lucene-Solr-Check-master/698/
: >> 
: >> 1 tests failed.
: >> FAILED:  org.apache.solr.TestRandomDVFaceting.testRandomFaceting
: >> 
: >> Error Message:
: >> java.lang.AssertionError: mismatch: 'LJAC'!='EKYL' @ 
facet_counts/facet_fields/id/[2]
: >> 
: >> Stack Trace:
: >> java.lang.AssertionError: mismatch: 'LJAC'!='EKYL' @ 
facet_counts/facet_fields/id/[2]
: >>at 
__randomizedtesting.SeedInfo.seed([B758812B355044AA:BA30A1FE60A98C15]:0)
: >>at org.junit.Assert.fail(Assert.java:89)
: >>at 
org.apache.solr.TestRandomDVFaceting.doFacetTests(TestRandomDVFaceting.java:292)
: >>at 
org.apache.solr.TestRandomDVFaceting.doFacetTests(TestRandomDVFaceting.java:175)
: >>at 
org.apache.solr.TestRandomDVFaceting.testRandomFaceting(TestRandomDVFaceting.java:158)
: >>at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
: >>at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
: >>at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
: >>at java.base/java.lang.reflect.Method.invoke(Method.java:566)
: >>at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
: >>at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:942)
: >>at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:978)
: >>at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
: >>at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
: >>at org.junit.rules.RunRules.evaluate(RunRules.java:20)
: >>at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
: >>at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
: >>at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
: >>at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
: >>at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
: >>at org.junit.rules.RunRules.evaluate(RunRules.java:20)
: >>at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: >>at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
: >>at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:819)
: >>at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:470)
: >>at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:951)
: >>at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:836)
: >>at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:887)
: >>at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
: >>at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: >>at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
: >>at org.junit.rules.RunRules.evaluate(RunRules.java:20)
: >>at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
: >>at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: >>at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
: >>at 

jira patch "precommit" jenkins jobs? (don't seem to be running lately)

[Chris Hostetter]

Does anyone know / un derstand the current status of the "PreCommit" 
jenkins jobs that are suppose to run against jira issues in the "Path 
Available" status?


For example: I noticed this AM that even though SOLR-14870 was in the 
"Path Available" status all weekend (I didn't want to commit build changes 
on a friday afternoon) it never got a comment from the jenkins build bot 
regarding the patch -- when I went looking for the "PreCommit" build jobs 
in jenkins i found that they are marked "N/A" for last 
success/failure/durration -- which i believe means they haven't run at all 
since all the jenkins jobs were moved to ci-builds.apache.org?


https://ci-builds.apache.org/job/Lucene/
https://ci-builds.apache.org/job/Lucene/job/PreCommit-LUCENE-Build/
https://ci-builds.apache.org/job/Lucene/job/PreCommit-SOLR-Build/

...the descriptions of those jobs say they are run by the 
"PreCommit-Admin" job, but the links is a 404.  searching jenkinds jobs 
turns up a few other "PreCommit" jobs in other projects -- most are 
disabled, except for this "Atlas" one which has run somewhat recently -- 
but it looks like people were manually triggering it?


https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/

do any of hte jenkins admins/experts know what's the status of the 
infra/jira hooks to get these jobs working again?




-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: 8.6.3 Release

[Chris Hostetter]

FWIW: I followed the docs to update the Dockerfiles + TAGS for 8.6.3, and 
run tests; but since it's in a distinct github repo I don't think i can 
push to it?

so i creaed a GH issue w/patch...

https://github.com/docker-solr/docker-solr/issues/349



: Date: Tue, 6 Oct 2020 11:33:15 -0400
: From: Houston Putman 
: Reply-To: dev@lucene.apache.org
: To: Solr/Lucene Dev 
: Subject: Re: 8.6.3 Release
: 
: That is correct. 8.x docker builds have not been affected in any way.
: 
: On Tue, Oct 6, 2020 at 11:30 AM Cassandra Targett 
: wrote:
: 
: > I wanted to ask now that the 8.6.3 vote is underway - for the docker-solr
: > image, are the update instructions in the docker-solr repo still the same
: > for 8.x even though the build process has been moved to the main project
: > for 9.0? Meaning, to release the 8.6.3 image there’s no change from before,
: > right?
: >
: > I’m asking specifically about these instructions:
: >
: > https://github.com/docker-solr/docker-solr/blob/master/update.md
: > On Oct 1, 2020, 9:28 AM -0500, Jason Gerlowski ,
: > wrote:
: >
: > I've put together draft Release Notes for 8.6.3 here. [1] [2]. Can
: > someone please sanity check the summaries there when they get a
: > chance? Would appreciate the review.
: >
: > 8.6.3 is a bit interesting in that Lucene has no changes in this
: > bugfix release. As a result I had to omit the standard phrase in the
: > Solr release notes about there being additional changes at the Lucene
: > level, and change some of the wording in the Lucene announcement to
: > indicate the lack of changes. So that's something to pay particular
: > attention to, if someone can check my wording there.
: >
: > [1] https://cwiki.apache.org/confluence/display/SOLR/DRAFT-ReleaseNote863
: > [2]
: > https://cwiki.apache.org/confluence/display/LUCENE/DRAFT-ReleaseNote863
: >
: > On Wed, Sep 30, 2020 at 10:57 AM Jason Gerlowski 
: > wrote:
: >
: >
: > The only one that was previously mentioned as a blocker was
: > SOLR-14835, but from the comments on the ticket it looks like it ended
: > up being purely a cosmetic issue. Andrzej left a comment there
: > suggesting that we "address" this with documentation for 8.6.3 but
: > otherwise leave it as-is.
: >
: > So it looks like we're unblocked on starting the release process.
: > Will begin the preliminary steps this afternoon.
: >
: > On Tue, Sep 29, 2020 at 3:40 PM Cassandra Targett 
: > wrote:
: >
: >
: > It looks to me like everything for 8.6.3 is resolved now (
: > https://issues.apache.org/jira/projects/SOLR/versions/12348713), and it
: > seems from comments in SOLR-14897 and SOLR-14898 that those fixes make a
: > Jetty upgrade less compelling to try.
: >
: > Are there any other issues not currently marked for 8.6.3 we’re waiting
: > for before starting the RC?
: > On Sep 29, 2020, 12:04 PM -0500, Jason Gerlowski ,
: > wrote:
: >
: > That said, if someone can use 8.6.3, what’s stopping them from going to
: > 8.7 when it’e released?
: >
: >
: > The same things that always stop users from going directly to the
: > latest-and-greatest: fear of instability from new minor-release
: > features, reliance on behavior changed across minor versions, breaking
: > changes on Lucene elements that don't guarantee backcompat (e.g.
: > SOLR-14254), security issues in later versions (new libraries pulled
: > in with vulns), etc. There's lots of reasons a given user might want
: > to stick on 8.6.x rather than 8.7 (in the short/medium term).
: >
: > I'm ambivalent to whether we upgrade Jetty in 8.6.3 - as I said above
: > the worst of the Jetty issue should be mitigated by work on our end -
: > but I think there's a lot of reasons users might not upgrade as far as
: > we'd expect/like.
: >
: >
: > On Mon, Sep 28, 2020 at 2:05 PM Erick Erickson 
: > wrote:
: >
: >
: > For me, there’s a sharp distinction between changing a dependency in a
: > point release just because there’s a new version, and changing the
: > dependency because there’s a bug in it. That said, if someone can use
: > 8.6.3, what’s stopping them from going to 8.7 when it’e released? Would it
: > make more sense to do the upgrades for 8.7 and get that out the door rather
: > than backport?
: >
: > FWIW,
: > Erick
: >
: > On Sep 28, 2020, at 1:45 PM, Jason Gerlowski 
: > wrote:
: >
: > Hey all,
: >
: > I wanted to add 2 more blocker tickets to the list: SOLR-14897 and
: > SOLR-14898. These tickets (while bad bugs in their own right) are
: > especially necessary because they work around a Jetty buffer-reuse bug
: > (see SOLR-14896) that causes sporadic request failures once triggered.
: >
: > So that brings the list of 8.6.3 blockers up to: SOLR-14850,
: > SOLR-14835, SOLR-14897, and SOLR-14898. (Thanks David for the quick
: > work on SOLR-14768!)
: >
: > Additionally, should we also consider a Jetty upgrade for 8.6.3 in
: > light of the issue mentioned above? I know it's atypical for bug-fix
: > releases to change deps, but here the bug is serious and tied directly
: > 

Re: [JENKINS] Lucene » Lucene-Solr-Check-master - Build # 485 - Still Failing!

[Chris Hostetter]

https://issues.apache.org/jira/browse/SOLR-12987




: Date: Thu, 1 Oct 2020 17:27:53 + (UTC)
: From: Apache Jenkins Server 
: Reply-To: dev@lucene.apache.org
: To: bui...@lucene.apache.org
: Subject: [JENKINS] Lucene » Lucene-Solr-Check-master - Build # 485 - Still
: Failing!
: 
: Build: 
https://ci-builds.apache.org/job/Lucene/job/Lucene-Solr-Check-master/485/
: 
: All tests passed
: 
: Build Log:
: [...truncated 1544 lines...]
: BUILD FAILED in 1h 6m 21s
: 834 actionable tasks: 834 executed
: Build step 'Invoke Gradle script' changed build result to FAILURE
: Build step 'Invoke Gradle script' marked build as failure
: Archiving artifacts
: Recording test results
: Email was triggered for: Failure - Any
: Sending email for trigger: Failure - Any
: 

-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: bin/solr testing surprise with techproducts example

[Chris Hostetter]

: Question: What is the replacement for "cd solr ; ant dist server" usage?

AFAICT the the most straightwoard "adaptation" of...

$ cd solr && ant server && bin/solr -e SOMETHING

...seems to be (using gdub) ...

$ cd solr && gw dev && ./packaging/build/dev/bin/solr -e SOMETHING

...which i think using straight gradlew would be...

$ ./gradlew -p solr dev && solr/packaging/build/dev/bin/solr -e SOMETHING


...and i'm all in favor of adding that to to one of the help files if i'm 
not hte only one that finds that patten incredibly common when trying to 
do little mini tests.


: Observation: "cd solr ; bin/solr start -e techproducts" on master branch 
: (but not branch_8x) gives me an error. Is this a known issue already or 
: if not could someone try to reproduce the issue before a JIRA ticket is 
: opened?
: 
: ERROR: Error CREATEing SolrCore 'techproducts': Unable to create core 
: [techproducts] Caused by: [schema.xml] analyzer/tokenizer: missing 
: mandatory attribute 'class'

FWIW, that has bitten me several times, and IIUC it's because i'm 
switching back and forth from 8x to master w/o doing full "git clean", and 
when the muscle memory types "bin/solr" instead of 
"packaging/build/dev/bin/solr" it's running old 8x compiled/build solr 
against the master example configs.

-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Avoiding false-positives in multivalued field search with intervals?

[Chris Hostetter]

(caveat: i don't ever really understand what Intervals at hte lucene 
feature set stage)

: Yup - similar to what Alan suggested. I'd have to rewrite the (general
: text-to-query) query parser to only use intervals though. Still
: thinking about possible approaches to this.
...
: > You could set a very high position increment gap for multi-valued 
: > fields (Analyzer#getPositionIncrementGap) and perform something
: > like Intervals.maxWidth(Intervals.unordered(...), pos_gap-1) ?

I'm assuming form your response that the issue here is really that you 
want to *directly* support the syntax you mentioned...

: >> > > doc: field=["foo", "bar"]
: >> > > query: field:(foo AND bar)

...and identify *when* the parser encouters a "boolean" expresion 
preceeded by the "fieldName:" syntax, and *then* treat thta special.

ie: this seems 100% like a query parser question, and not at all as a 
"what does the query structure look like ater parsing" question.

Because if you can adjust your parser syntax, this literallyly just 
becomes:  ' field:"foo bar"~N '   ...  where N is the positionIncrementGap 
on your analyzer ... OR ... ' field:"foo bar" ' ... if you call 
setPhraseSlop on your QueryParser.

i *THINK* the crux of your question/problem is that -- from the point of 
view of the QueryParserBase/BooleanQueryNodeBuilder, these 2 input strings 
are treated identically by the time any "subclass" has a chance to do anything 
interesting with them...

field:(foo AND bar)
field:foo AND field:bar

...so you can't for instance, build an Interval / sloppy Phrase query from 
the first, while building a 2 clause boolean query from the second.

So maybe the "solution" (at least for the flexible parser ... IIUC, I 
haven't used it much) would be for BooleanQueryNode to carry some metadata 
indicating that there was a "fieldName:" prefix on it, so that the 
BooleanQueryNodeBuilder can choose to use that information to do something 
"special" if the "List clauses" are all simple TermNodes (in 
the same field)

?


-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: QueryParser - proposed change may break existing queries.

[Chris Hostetter]

: And as I understand it, current behavior is the silent misinterpretation.
: To me, the failure to require a space after the regex (and either not
: become a regex in that case or complain about invalid regex) might be
: considered a bug...

I would agree ...

: >> However, today people can search for :
: >>/foo.com/index.html
: >> and not get an error. The searcher may think this is a query for a URL
: >> but it's actually parsed as a regex "foo.com" ORed with a term query.

... i didn't realize that was happening.  To me that seems like it should 
definitely be considered a bug, and the "regex" branch of the grammer 
shouldn't be used if there is any unexpected characters after the closing 
"/" ... the current behavior Mark is describgin seems analogous to the 
grammer assuming "WESS ANDERSON" should be parsed as "WESS +DERSON"

: > You could avoid (some of?) these problems by supporting /(?i)foo/ 
: > instead of /foo/i 
: 
: I like this idea. The only downside is that folks will tend to think
: it's a full Java Pattern and try other options. :)

If they try to use any other options then 'i' we throow a ParseException 





-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

The "supported" way to run gradle -- was: Re: [jira] [Commented] (LUCENE-9495) Ref Guide gradle buildSite ignores version parameter

[Chris Hostetter]

Spinning this off of a side comment Dawid made in a jira...

: ... Hoss -- you absolutely should use the provided wrapper script, not 
: your system's gradle. 
: {code} 
: hossman@slate:~/lucene/dev/solr/solr-ref-guide [j11] [master] $ gradle 
buildSite -PsolrGuideVersion=9.0 
: {code} 
: This is important as those startup shell scripts in the repo have 
: additional stuff added on top of them compared to stock gradle.

I have 'gradle' aliased to 'gw' ( aka: 'gdub" => 
https://github.com/gdubw/gdub ) ... which i thought was a recomendation I 
had seen from Dawid but i'm not finidng it now, so i honestly have no idea 
where I leaned about it.

IIUC this is "doing the right thing" as far as running gradlew if it 
exists -- but now i'm not so sure? ... in the output/code block Dawid 
replied to above, the very first line of output was 'gw' saying...

>> Using gradle at '/home/hossman/lucene/dev/gradlew' to run buildfile 
>> '/home/hossman/lucene/dev/solr/solr-ref-guide/build.gradle':

...but is that "enough" ? or is there context/options that may not be 
getting picked up unless i explicitly run './gradlew -p 
solr/solr-ref-guide ... ' from the root level checkout?


As a novice gradle user these questions put me in the mind of a "new dev" 
coming to lucene, and I thought "Let's check the README!" ... where I see 
instructions to use './gradlew' mentioned (and pointers to './gradlew 
help') but nothing that really jumps out at me at screams "It's really 
important to use './gradlew' not 'gradle' (or 'gw') and here's why: ..." 
-- should there be?

Lastly: Is there anyway to make our 'build.gradle' files "fail" if someone 
does *NOT* use our './gradlew' (ie: maybe set a special property in 
gradlew that our build.gradle file can look for and fail if unset?


-Hoss

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Annoying but harmless exceptions due to filepermissions when running tests

[Chris Hostetter]

: Hmmm, that’s kind of an dilemma then. Are you saying that 
: he test can see that the directory appears writable then tries
: to write to it then gets tripped up by the framework?
: 
: Seems to me that a test that tries to write, thinks it can and then
: can’t should fail anyway.
: 
: Well, I don’t think there are very many tests that have this problem
: anyway, so maybe I can examine them one-by-one and not 
: introduce new failures...

You keep using the phrase "the test" in the context of (trying to) create 
these directories ("userfiles" and "filestore") ... the "test CODE" isn't 
making any choices about trying to write those files -- the choice is 
being made by the "CoreContainer CODE".

These features were added with the explicit implementation choice to _TRY_ 
to write the "usersfiles" (and/or "filestore") directory to "solr home" IF 
POSSIBLE, and if so then enable a bunch of features -- if NOT then log a 
WARNing and don't enable those features.

So what you're seeing here isn't an artifact/result of any particular 
choices "a test" or "the test" makes -- it's a concious choice of the 
developer who added this feature to solr.  These WARN messages that show 
up in tests where the solr home dir isn't writable (which is actaully the 
vast majority of tests because of how the test framework works) are the 
same types of WARN messages that a "real" solr deployment might get if 
their solr home dir isn't wriable (ie: maybe the use ${solr.data.dir} to 
point to a diff drive).



: 
: > On Aug 31, 2020, at 1:29 PM, Chris Hostetter  
wrote:
: > 
: > 
: > Some tests "create" a new solr home dir and copy config files there, but 
: > you'll see this type of WARN logging for any test that just uses the test 
: > configs "in place" because of how the code is designed to _try_ and create 
: > a userfiles directory in the solr home if it's writable.
: > 
: > 
: > : Date: Sat, 29 Aug 2020 09:25:17 -0400
: > : From: Erick Erickson 
: > : Reply-To: dev@lucene.apache.org
: > : To: dev@lucene.apache.org
: > : Subject: Re: Annoying but harmless exceptions due to filepermissions when
: > : running tests
: > : 
: > : Well, as Uwe and I discussed offline, he’s right and I’m wrong.
: > : 
: > : In CoreContainer [364] there’s code like this:
: > : 
: > : Path userFilesPath = return solrHome.resolve("userfiles"); // TODO make 
configurable on cfg?
: > : try {
: > :   Files.createDirectories(userFilesPath); // does nothing if already 
exists
: > : } catch (Exception e) {
: > :   log.warn("Unable to create [{}].  Features requiring this directory may 
fail.", userFilesPath, e);
: > : }
: > : 
: > : So I assumed it would happen on most every test, at least in cloud mode. 
But when I tried to make it happen on a different test, there was no exception.
: > : 
: > : I’ll have to poke some more to see what’s really happening…
: > : 
: > : Never Mind….
: > : 
: > : > On Aug 29, 2020, at 8:59 AM, Uwe Schindler  wrote:
: > : > 
: > : > Hi,
: > : > 
: > : > this is a bug in the test! It should never ever modify files outside 
its sandbox. It should not even modify files in build directory. It is fully 
valid to reject those writes - has nothing to do with users, it's just 
forbidden by the test framework. Modifying this file is harmful, as it may 
affect later tests.
: > : > 
: > : > So the correct way is to copy those files to the solr container running 
inside test's sandbox. That's one of the main advantages of the Test sandbox: 
No write access anywhere outside the test, see policy file.
: > : > 
: > : > Uwe
: > : > 
: > : > -
: > : > Uwe Schindler
: > : > Achterdiek 19, D-28357 Bremen
: > : > https://www.thetaphi.de
: > : > eMail: u...@thetaphi.de
: > : > 
: > : >> -Original Message-
: > : >> From: Erick Erickson 
: > : >> Sent: Saturday, August 29, 2020 2:54 PM
: > : >> To: dev@lucene.apache.org
: > : >> Subject: Annoying but harmless exceptions due to filepermissions when 
running
: > : >> tests
: > : >> 
: > : >> These exceptions are handled in the code and don’t affect running 
tests, but
: > : >> they can be a distraction when trying to figure out what’s causing a 
failure.
: > : >> When CoreContainer is being initialized, these two paths get 
“Permission
: > : >> Denied” errors since they try to create directories/files.
: > : >> 
: > : >> java.security.AccessControlException: access denied 
("java.io.FilePermission"
: > : >> 
"/Users/Erick/apache/solrJiras/master/solr/core/build/resources/test/solr/filest
: > : >> o

Re: Annoying but harmless exceptions due to filepermissions when running tests

[Chris Hostetter]

Some tests "create" a new solr home dir and copy config files there, but 
you'll see this type of WARN logging for any test that just uses the test 
configs "in place" because of how the code is designed to _try_ and create 
a userfiles directory in the solr home if it's writable.


: Date: Sat, 29 Aug 2020 09:25:17 -0400
: From: Erick Erickson 
: Reply-To: dev@lucene.apache.org
: To: dev@lucene.apache.org
: Subject: Re: Annoying but harmless exceptions due to filepermissions when
: running tests
: 
: Well, as Uwe and I discussed offline, he’s right and I’m wrong.
: 
: In CoreContainer [364] there’s code like this:
: 
: Path userFilesPath = return solrHome.resolve("userfiles"); // TODO make 
configurable on cfg?
: try {
:   Files.createDirectories(userFilesPath); // does nothing if already exists
: } catch (Exception e) {
:   log.warn("Unable to create [{}].  Features requiring this directory may 
fail.", userFilesPath, e);
: }
: 
: So I assumed it would happen on most every test, at least in cloud mode. But 
when I tried to make it happen on a different test, there was no exception.
: 
: I’ll have to poke some more to see what’s really happening…
: 
: Never Mind….
: 
: > On Aug 29, 2020, at 8:59 AM, Uwe Schindler  wrote:
: > 
: > Hi,
: > 
: > this is a bug in the test! It should never ever modify files outside its 
sandbox. It should not even modify files in build directory. It is fully valid 
to reject those writes - has nothing to do with users, it's just forbidden by 
the test framework. Modifying this file is harmful, as it may affect later 
tests.
: > 
: > So the correct way is to copy those files to the solr container running 
inside test's sandbox. That's one of the main advantages of the Test sandbox: 
No write access anywhere outside the test, see policy file.
: > 
: > Uwe
: > 
: > -
: > Uwe Schindler
: > Achterdiek 19, D-28357 Bremen
: > https://www.thetaphi.de
: > eMail: u...@thetaphi.de
: > 
: >> -Original Message-
: >> From: Erick Erickson 
: >> Sent: Saturday, August 29, 2020 2:54 PM
: >> To: dev@lucene.apache.org
: >> Subject: Annoying but harmless exceptions due to filepermissions when 
running
: >> tests
: >> 
: >> These exceptions are handled in the code and don’t affect running tests, 
but
: >> they can be a distraction when trying to figure out what’s causing a 
failure.
: >> When CoreContainer is being initialized, these two paths get “Permission
: >> Denied” errors since they try to create directories/files.
: >> 
: >> java.security.AccessControlException: access denied 
("java.io.FilePermission"
: >> 
"/Users/Erick/apache/solrJiras/master/solr/core/build/resources/test/solr/filest
: >> ore" "write”)
: >> 
: >> java.security.AccessControlException: access denied 
("java.io.FilePermission"
: >> 
"/Users/Erick/apache/solrJiras/master/solr/core/build/resources/test/solr/userf
: >> iles" "write”)
: >> 
: >> In both cases, the code logs a warning like "Features requiring this 
directory
: >> may fail”.
: >> 
: >> “build” is permitted this way, so I guess gradle runs as some other user?
: >> 
: >> drwxr-xr-x  11 Erick  staff352 Aug 28 09:30 build
: >> 
: >> Any hints on an easy way to avoid these? It’s not worth much effort I don’t
: >> think since they’re handled, but if it’s easy I’d like to not see them.
: >> 
: >> 
: >> 
: >> -
: >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
: >> For additional commands, e-mail: dev-h...@lucene.apache.org
: > 
: > 
: > -
: > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
: > For additional commands, e-mail: dev-h...@lucene.apache.org
: > 
: 
: 
: -
: To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
: For additional commands, e-mail: dev-h...@lucene.apache.org
: 
: 

-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: ASF Jenkins moved to new URL

[Chris Hostetter]

Thanks for the ping Uwe, reports tool udated iwth new url...

https://github.com/hossman/jenkins-reports/commit/68d42574fb4b306254653f7f72ae7ff01d6c3992

...it seemed to work locally, public reports should start showing correct 
data in a few hours.

(Although i don't see any 'PreCommit' jobs yet showing up in the build 
log? ... my reports are designed to ignore those since thery are expected 
to have failures, but i'm not sure if the "ignore" rule will work w/o 
seeing what their new job names/urls look like)



: Date: Thu, 6 Aug 2020 18:08:55 +0200
: From: Uwe Schindler 
: To: Chris Hostetter 
: Cc: dev@lucene.apache.org
: Subject: ASF Jenkins moved to new URL
: 
: Hi Hossman,
: 
: as you are doing the statistics on Jenkins, I just wanted to give you a
: ping, because the build servers of ASF changed to Cloudbees. It's still
: jenkins, but different structure.
: 
: I moved all jobs over yesterday, there are still some issue (a reboot is
: required to enable a plugin needed for "Nightly" jobs and a permissions
: problem for refguide must be fixed. Nevertheless, the API should be
: compatible, but there are no veiws anymore. The jobs can be found here:
: https://ci-builds.apache.org/job/Lucene/
: 
: It's a "folder" and no "view" anymore (only at ASF, policeman is still
: different). I think your script should be able to handle that, but would be
: good to check. The new structure also have its own slave nodes (lucene1 and
: lucene2 is private to jobs from the lucene folder). The old jenkins is still
: visible, but all jobs are disabled there - as no node is there anymore to
: execute them. It will be switched off on Aug 15th.
: 
: Uwe
: 
: -
: Uwe Schindler
: Achterdiek 19, D-28357 Bremen
: https://www.thetaphi.de
: eMail: u...@thetaphi.de
: 
: 
: 

-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

RE: Policeman Jenkins lost it's "/view/Lucene-Solr/"

[Chris Hostetter]

: it seem to got lost on last Update. I was on vacation last week, so missed 
your mail.
: 
: I added it back,

thanks! ... also: based on the recent jenkins emails it looks like your 
disk is full?

(welcome back!)

: Uwe
: 
: -
: Uwe Schindler
: Achterdiek 19, D-28357 Bremen
: https://www.thetaphi.de
: eMail: u...@thetaphi.de
: 
: > -Original Message-
: > From: Chris Hostetter 
: > Sent: Monday, July 13, 2020 6:05 PM
: > To: Uwe Schindler 
: > Cc: Lucene Dev 
: > Subject: Re: Policeman Jenkins lost it's "/view/Lucene-Solr/"
: > 
: > 
: > Uwe? ... you around? can you please re-add this jenkins view?
: > 
: > : Date: Thu, 2 Jul 2020 17:36:25 -0700 (MST)
: > : From: Chris Hostetter 
: > : To: Uwe Schindler 
: > : Cc: Lucene Dev 
: > : Subject: Policeman Jenkins lost it's "/view/Lucene-Solr/"
: > :
: > :
: > : Uwe: the "/view/Lucene-Solr/" View that my reports were using to restrict 
the
: > : jobs it checked on your jenkins server (to only lucene/solr jobs) seems to
: > : have been removed...
: > :
: > :   https://jenkins.thetaphi.de/view/Lucene-Solr/
: > :
: > : ...as a result, my reports evidently haven't detected any failures (or
: > : successes) from your jenkins jobs in a while...
: > :
: > :   http://fucit.org/solr-jenkins-reports/
: > :
: > : ...can you please add that view back?
: > :
: > :
: > : -Hoss
: > : http://www.lucidworks.com/
: > :
: > 
: > -Hoss
: > http://www.lucidworks.com/
: 
: 

-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Policeman Jenkins lost it's "/view/Lucene-Solr/"

[Chris Hostetter]

Uwe? ... you around? can you please re-add this jenkins view?

: Date: Thu, 2 Jul 2020 17:36:25 -0700 (MST)
: From: Chris Hostetter 
: To: Uwe Schindler 
: Cc: Lucene Dev 
: Subject: Policeman Jenkins lost it's "/view/Lucene-Solr/"
: 
: 
: Uwe: the "/view/Lucene-Solr/" View that my reports were using to restrict the
: jobs it checked on your jenkins server (to only lucene/solr jobs) seems to
: have been removed...
: 
:   https://jenkins.thetaphi.de/view/Lucene-Solr/
: 
: ...as a result, my reports evidently haven't detected any failures (or
: successes) from your jenkins jobs in a while...
: 
:   http://fucit.org/solr-jenkins-reports/
: 
: ...can you please add that view back?
: 
: 
: -Hoss
: http://www.lucidworks.com/
: 

-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS-Experimental-GC] Lucene-Solr-8.6-MacOSX (64bit/jdk-14.0.1) - Build # 100 - Still Unstable!

[Chris Hostetter]

: Note: it is expected for MultiThreadedOCPTest to fail from time to time on
: the 8x branch as SOLR-14546
:  was only fixed in 9.0.
: Such failures would be unexpected on master.

If there is no intention of fixing this on branch_8x (either because of 
backcompat concerns, or whatever other concerns may exist...) but the 
problem is "known" to cause test failures then the test should be 
disabled/removed from branch_8x.


-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Policeman Jenkins lost it's "/view/Lucene-Solr/"

[Chris Hostetter]

Uwe: the "/view/Lucene-Solr/" View that my reports were using to restrict 
the jobs it checked on your jenkins server (to only lucene/solr jobs) 
seems to have been removed...


https://jenkins.thetaphi.de/view/Lucene-Solr/

...as a result, my reports evidently haven't detected any failures (or 
successes) from your jenkins jobs in a while...


http://fucit.org/solr-jenkins-reports/

...can you please add that view back?


-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: missing javascript support -- was: Re: [JENKINS-Experimental-GC] Lucene-Solr-8.x-Linux (64bit/jdk-15-ea+24) - Build # 3304 - Unstable!

[Chris Hostetter]

: It looks like jdk15 dropped support for Javascript (憐).

Weird.  I knew they had deprecated the Nashorn engine, but i thought there 
was a replacement javascript engine?

I guess now it's an external / third-party dep?...

https://golb.hplar.ch/2020/04/java-javascript-engine.html

: >...we can/should probably make this specific test smarter about
: >"assuming" 
: >what ScriptEngines are available ... but it still raises the question:

master: 924e7d5cd6e29fb736fb462e09fd0a1a05a7649a
8x: 83444b220fca07d40101e6fe9a5066c920b41803


-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

missing javascript support -- was: Re: [JENKINS-Experimental-GC] Lucene-Solr-8.x-Linux (64bit/jdk-15-ea+24) - Build # 3304 - Unstable!

[Chris Hostetter]

Uwe: TestConfigSetsAPI.testUploadWithScriptUpdateProcessor started failing 
on many of your jenkins builds ~ a week ago?

Digging into the logs i see this...

   [junit4]   2> 652490 ERROR (qtp2055916805-15908) [n:127.0.0.1:45223_solr 
] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Error 
CREATEing SolrCore 'newcollection2_shard1_replica_n1': Unable to create core 
[newcollection2_shard1_replica_n1] Caused by: No ScriptEngine found by name: 
javascript -- supported names: 
   [junit4]   2>at 
org.apache.solr.core.CoreContainer.create(CoreContainer.java:1262)
   [junit4]   2>at 
org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:95)
...
   [junit4]   2> Caused by: org.apache.solr.common.SolrException: Unable to 
initialize scripts: No ScriptEngine found by name: javascript -- supported 
names: 
   [junit4]   2>at 
org.apache.solr.core.SolrCore.(SolrCore.java:1068)
   [junit4]   2>at 
org.apache.solr.core.SolrCore.(SolrCore.java:905)
   [junit4]   2>at 
org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1314)
   [junit4]   2>... 40 more
   [junit4]   2> Caused by: org.apache.solr.common.SolrException: Unable to 
initialize scripts: No ScriptEngine found by name: javascript -- supported 
names: 
   [junit4]   2>at 
org.apache.solr.update.processor.StatelessScriptUpdateProcessorFactory.inform(StatelessScriptUpdateProcessorFactory.java:243)
   [junit4]   2>at 
org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:618)
   [junit4]   2>at 
org.apache.solr.core.SolrCore.(SolrCore.java:1021)
   [junit4]   2>... 42 more
   [junit4]   2> Caused by: org.apache.solr.common.SolrException: No 
ScriptEngine found by name: javascript -- supported names: 
   [junit4]   2>at 
org.apache.solr.update.processor.StatelessScriptUpdateProcessorFactory.initEngines(StatelessScriptUpdateProcessorFactory.java:287)
   [junit4]   2>at 
org.apache.solr.update.processor.StatelessScriptUpdateProcessorFactory.inform(StatelessScriptUpdateProcessorFactory.java:239)

...we can/should probably make this specific test smarter about "assuming" 
what ScriptEngines are available ... but it still raises the question: why 
isn't javascript available on your jenkins box anymore uwe?  I thought 
the last time i looked that java ScriptEngine spec said "javascript" was a 
required implementatoin?




: Date: Fri, 5 Jun 2020 16:57:55 + (UTC)
: From: Policeman Jenkins Server 
: Reply-To: dev@lucene.apache.org
: To: bui...@lucene.apache.org
: Subject: [JENKINS-Experimental-GC] Lucene-Solr-8.x-Linux (64bit/jdk-15-ea+24)
: - Build # 3304 - Unstable!
: 
: Build: https://jenkins.thetaphi.de/job/Lucene-Solr-8.x-Linux/3304/
: Java: 64bit/jdk-15-ea+24 -XX:+UseCompressedOops 
-XX:+UnlockExperimentalVMOptions -XX:+UseZGC
: 
: 3 tests failed.
: FAILED:  
org.apache.solr.cloud.TestConfigSetsAPI.testUploadWithScriptUpdateProcessor
: 
: Error Message:
: Error from server at http://127.0.0.1:34935/solr: Underlying core creation 
failed while creating collection: newcollection2
: 
: Stack Trace:
: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:34935/solr: Underlying core creation failed 
while creating collection: newcollection2
:   at 
__randomizedtesting.SeedInfo.seed([357851D24C39FCBD:5A274DAD57085F4B]:0)
:   at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:663)
:   at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:265)
:   at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
:   at 
org.apache.solr.client.solrj.impl.LBSolrClient.doRequest(LBSolrClient.java:368)
:   at 
org.apache.solr.client.solrj.impl.LBSolrClient.request(LBSolrClient.java:296)
:   at 
org.apache.solr.client.solrj.impl.BaseCloudSolrClient.sendRequest(BaseCloudSolrClient.java:1147)
:   at 
org.apache.solr.client.solrj.impl.BaseCloudSolrClient.requestWithRetryOnStaleState(BaseCloudSolrClient.java:910)
:   at 
org.apache.solr.client.solrj.impl.BaseCloudSolrClient.request(BaseCloudSolrClient.java:842)
:   at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1290)
:   at 
org.apache.solr.cloud.TestConfigSetsAPI.createCollection(TestConfigSetsAPI.java:578)
:   at 
org.apache.solr.cloud.TestConfigSetsAPI.testUploadWithScriptUpdateProcessor(TestConfigSetsAPI.java:364)
:   at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
:   at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
:   at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
:   at java.base/java.lang.reflect.Method.invoke(Method.java:564)
:   at 

Re: [VOTE] Solr to become a top-level Apache project (TLP)

[Chris Hostetter]

: Therefore, the proposal is to separate Solr from under Lucene TLP, and
: make it a TLP on its own. The initial structure of the new PMC,
: committer base, git repositories and other managerial aspects can be
: worked out during the process if the decision passes.
: 
: Please indicate one of the following (see [1] for guidelines):
: 
: [ ] +1 - yes, I vote for the proposal
: [ ] -1 - no, I vote against the proposal

-1 

(non-binding)


-Hoss


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: PLEASE READ! BadApple report. Last week was horrible!

[Chris Hostetter]

: And FWIW, I beasted one of the failing suites last night _without_ 
: Mike’s changes and didn’t get any failures so I can’t say anything about 
: whether Mike’s changes helped or not.

IIUC McCandless's failure only affects you if you use the "jenkins" test 
data file (the really big wikipedia dump) ... see the jira he mentioned 
for details.



-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Switching Solr to HTTPS

[Chris Hostetter]

: I would like to try and address the issue of switching from http to https as 
referenced
: in https://issues.apache.org/jira/browse/SOLR-12182

I would suggest asking these questions (both about hte validity of 
the patch file, and about possible work arounds) directly in the jira you 
mentioned.

IIUC: If you establish SSL when you first create your cluster, prior to 
creating any collections, this bug should be a non-issue, which is 
probably why it hasn't seen much action/comments since Anshum created it.


-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Lucene 9.0 Java module system support

[Chris Hostetter]

: If the changes I proposed are still viewed as having too many downstream
: impacts, my fallback position will be to patch the jars. This involves
: using the gradle import system to get the jars from Maven, then using a
: patch script to manually unzip the jars, move the offending classes into
: other jars which share the same package name and rezip. So far, I've been

I'm no expert here, but i trust that Uwe is, and i feel like your followup 
questions/suggests have still avoided his primary point about *why* 
Lucene/Solr hasn't attempted jigsaw modulariation...

: >>> There is currently some preparatory things to move forward with modules,
: >>> so although you might be able to actually compile Lucene with module 
system
: >>> (by limiting to a subset of JAR files), it currently won’t work
: >>> cross-module due to the way how it handles ServiceLoader interfaces
: >>> (codecs, postings formats, analyzers, see
: >>> https://issues.apache.org/jira/browse/LUCENE-9281). The only way to
: >>> make it work at runtime is to put all of Lucene into one module.

...so, IIUC, even if you patch the *current* jars, any Lucene code you use 
that depends on SPI (like analysis chains, codecs, etc...) isn't going to 
work unless follow Uwe's primary suggestion for folks who care about 
modules...

: >>> Th general recommendation is to combine all required Lucene libraries
: >>> into a separate JAR file during the maven / gradle build (e.g. using the
: >>> Maven Shade plugin). Keep in mind that Lucene is also not suitable for use


-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: 8.5 release

[Chris Hostetter]

: I’ve created a branch for the 8.5 release (`branch_8_5`) and pushed it 
: to the apache repository.  We’re now at feature freeze, so only bug 
: fixes should be pushed to the branch.

I'm a little confused where folks should put stuff in CHANGES.txt right 
now if it's *NOT* something ready to backport all the way to branch_8_5.

specifically: branch_8x now has an 8.6 section in CHANGES.txt, but master 
doesn't ... so if i have a feature (or low priority bug fix that I don't 
want to rush into 8.5 w/o more review) I'm not sure where/how to record it 
w/o having weird merge conflicts (or causing weird merge conflicts later0


-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: "Cannot start mini solr cloud cluster" errors

[Chris Hostetter]

https://lists.apache.org/thread.html/r830f54ed63361fcb3dbe13669d60cabbe87bca0309f3b5076c15d6af%40%3Cdev.lucene.apache.org%3E

: Date: Tue, 3 Mar 2020 17:24:23 -0700
: From: Erick Erickson 
: Reply-To: dev@lucene.apache.org
: To: dev@lucene.apache.org
: Subject: "Cannot start mini solr cloud cluster" errors
: 
: Here's one of the stack traces, anyone have a clue? Can't tell if it
: reproduces, my internet connections are iffy for the next few days:
: 
: Here's a seed, not sure whether it reproduces or not, train's pulling out:
: 
: ant test  -Dtestcase=LegacyFieldFacetCloudTest
: -Dtests.seed=CFEFCCB6130E91BE -Dtests.slow=true
: -Dtests.badapples=trueDtests.locale=fr-CF
: -Dtests.timezone=Asia/Nicosia -Dtests.asserts=true
: -Dtests.file.encoding=ISO-8859-1
: 
: Stack traces look like:
: 
: [junit4]   2> 28559 INFO  (jetty-launcher-25-thread-2) [ ]
: o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
:[junit4]   2> 28845 INFO  (zkConnectionManagerCallback-27-thread-1)
: [ ] o.a.s.c.c.ConnectionManager zkClient has connected
:[junit4]   2> 28850 INFO  (jetty-launcher-25-thread-1) [ ]
: o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
:[junit4]   2> 28890 INFO  (zkConnectionManagerCallback-29-thread-1)
: [ ] o.a.s.c.c.ConnectionManager zkClient has connected
:[junit4]   2> 28891 INFO  (jetty-launcher-25-thread-2) [ ]
: o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
:[junit4]   2> 28896 INFO  (jetty-launcher-25-thread-1) [ ]
: o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
:[junit4]   2> 28901 INFO  (jetty-launcher-25-thread-2) [ ]
: o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
:[junit4]   2> 29071 ERROR (jetty-launcher-25-thread-1) [ ]
: o.a.s.s.SolrDispatchFilter Could not start Solr. Check solr/home
: property and the logs
:[junit4]   2> 29071 ERROR (jetty-launcher-25-thread-1) [ ]
: o.a.s.c.SolrCore null:java.lang.IllegalArgumentException: String
: length must be a multiple of four.
:[junit4]   2> at
: org.apache.solr.common.util.Base64.base64ToByteArray(Base64.java:102)
:[junit4]   2> at
: org.apache.solr.util.CryptoKeys$RSAKeyPair.(CryptoKeys.java:369)
:[junit4]   2> at
: 
org.apache.solr.security.PublicKeyHandler.createKeyPair(PublicKeyHandler.java:59)
:[junit4]   2> at
: org.apache.solr.security.PublicKeyHandler.(PublicKeyHandler.java:43)
:[junit4]   2> at
: org.apache.solr.core.CoreContainer.(CoreContainer.java:342)
:[junit4]   2> at
: org.apache.solr.core.CoreContainer.(CoreContainer.java:330)
:[junit4]   2> at
: 
org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:262)
:[junit4]   2> at
: org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:183)
:[junit4]   2> at
: org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:134)
:[junit4]   2> at
: 
org.eclipse.jetty.servlet.ServletHandler.lambda$initialize$0(ServletHandler.java:751)
:[junit4]   2> at
: 
java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
:[junit4]   2> at
: 
java.base/java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:734)
:[junit4]   2> at
: 
java.base/java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:734)
:[junit4]   2> at
: 
java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:658)
:[junit4]   2> at
: org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:744)
:[junit4]   2> at
: 
org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1448)
:[junit4]   2> at
: 
org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1507)
:[junit4]   2> at
: 
org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1164)
:[junit4]   2> at
: 
org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1003)
:[junit4]   2> at
: 
org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:459)
:[junit4]   2> at
: 
org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:382)
:[junit4]   2> at
: 
org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:193)
:[junit4]   2> at
: 
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
:[junit4]   2> at
: 
org.apache.solr.client.solrj.embedded.JettySolrRunner.retryOnPortBindFailure(JettySolrRunner.java:565)
:[junit4]   2> at
: 
org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:504)
:[junit4]   2> at
: 
org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:472)
:[junit4]   2> at
: 

Re: [JENKINS] Lucene-Solr-master-Windows (64bit/jdk-13.0.2) - Build # 8538 - Still Unstable!

[Chris Hostetter]

Anybody else remotely interested in giving a fuck about the systemic 
Windows Solr test failures that started on Feb-24 ? ... becuase I'm kind 
of over the whole "being the jenkins babysitter" job -- it was getting 
too sysiphian.

Oh hey look, all of the failing tests have an ERROR in the logs with a 
stack trace that refers to classes that were modified on Feb-24...

   [junit4]   2> 13804 ERROR (jetty-launcher-8-thread-1) [ ] 
o.a.s.s.SolrDispatchFilter Could not start Solr. Check solr/home property and 
the logs
   [junit4]   2> 13837 ERROR (jetty-launcher-8-thread-1) [ ] 
o.a.s.c.SolrCore null:java.lang.IllegalArgumentException: String length must be 
a multiple of four.
   [junit4]   2>at 
org.apache.solr.common.util.Base64.base64ToByteArray(Base64.java:102)
   [junit4]   2>at 
org.apache.solr.util.CryptoKeys$RSAKeyPair.(CryptoKeys.java:369)
   [junit4]   2>at 
org.apache.solr.security.PublicKeyHandler.createKeyPair(PublicKeyHandler.java:59)
   [junit4]   2>at 
org.apache.solr.security.PublicKeyHandler.(PublicKeyHandler.java:43)
   [junit4]   2>at 
org.apache.solr.core.CoreContainer.(CoreContainer.java:341)
   [junit4]   2>at 
org.apache.solr.core.CoreContainer.(CoreContainer.java:329)





: Date: Mon, 2 Mar 2020 14:57:50 + (UTC)
: From: Policeman Jenkins Server 
: Reply-To: dev@lucene.apache.org
: To: bui...@lucene.apache.org
: Subject: [JENKINS] Lucene-Solr-master-Windows (64bit/jdk-13.0.2) - Build #
: 8538 - Still Unstable!
: 
: Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/8538/
: Java: 64bit/jdk-13.0.2 -XX:+UseCompressedOops -XX:+UseG1GC
: 
: 96 tests failed.
: FAILED:  
junit.framework.TestSuite.org.apache.solr.analytics.legacy.LegacyNoFacetCloudTest
: 
: Error Message:
: Error starting up MiniSolrCloudCluster
: 
: Stack Trace:
: java.lang.Exception: Error starting up MiniSolrCloudCluster
:   at __randomizedtesting.SeedInfo.seed([5B1EF57CA04E8DEB]:0)
:   at 
org.apache.solr.cloud.MiniSolrCloudCluster.checkForExceptions(MiniSolrCloudCluster.java:656)
:   at 
org.apache.solr.cloud.MiniSolrCloudCluster.(MiniSolrCloudCluster.java:308)
:   at 
org.apache.solr.cloud.SolrCloudTestCase$Builder.build(SolrCloudTestCase.java:220)
:   at 
org.apache.solr.cloud.SolrCloudTestCase$Builder.configure(SolrCloudTestCase.java:210)
:   at 
org.apache.solr.analytics.legacy.LegacyAbstractAnalyticsCloudTest.setupCollection(LegacyAbstractAnalyticsCloudTest.java:50)
:   at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
:   at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
:   at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
:   at java.base/java.lang.reflect.Method.invoke(Method.java:567)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:882)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
:   at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
:   at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
:   at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
:   at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
:   at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
:   at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
:   at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
:   at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
:   at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
:   at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
:   at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
:   at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
:   at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
:   at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
:   at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
:   at 

Re: [jira] [Commented] (SOLR-13778) Windows JDK SSL Test Failure trend: SSLException: Software caused connection abort: recv failed

[Chris Hostetter]

If we were in a movie, now is when Dawid would be suddenly and 
unexpectedly be targetted by assassins.


: Date: Thu, 19 Dec 2019 21:38:00 + (UTC)
: From: "Dawid Weiss (Jira)" 
: To: hoss...@apache.org
: Subject: [jira] [Commented] (SOLR-13778) Windows JDK SSL Test Failure trend:
: SSLException: Software caused connection abort: recv failed
: 
: 
: [ 
https://issues.apache.org/jira/browse/SOLR-13778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17000418#comment-17000418
 ] 
: 
: Dawid Weiss commented on SOLR-13778:
: 
: 
: I know what it is. I'll try to reproduce first to showcase why it's 
happening. :)
: 
: > Windows JDK SSL Test Failure trend: SSLException: Software caused 
connection abort: recv failed
: > 
---
: >
: > Key: SOLR-13778
: > URL: https://issues.apache.org/jira/browse/SOLR-13778
: > Project: Solr
: >  Issue Type: Bug
: >  Security Level: Public(Default Security Level. Issues are Public) 
: >Reporter: Chris M. Hostetter
: >Priority: Major
: > Attachments: dumps-LegacyCloud.zip, logs-2019-12-12-1.zip, 
recv-multiple-2019-12-18.zip
: >
: >
: > Now that Uwe's jenkins build has been correctly reporting it's build 
results for my [automated 
reports|http://fucit.org/solr-jenkins-reports/failure-report.html] to pick up, 
I've noticed a pattern of failures that indicate a definite problem with using 
SSL on Windows (even with java 11.0.4
: >  )
: >  The symptommatic stack traces all contain...
: > {noformat}
: > ...
: >[junit4]> Caused by: javax.net.ssl.SSLException: Software caused 
connection abort: recv failed
: >[junit4]>at 
java.base/sun.security.ssl.Alert.createSSLException(Alert.java:127)
: > ...
: >[junit4]> Caused by: java.net.SocketException: Software caused 
connection abort: recv failed
: >[junit4]>at 
java.base/java.net.SocketInputStream.socketRead0(Native Method)
: > ...
: > {noformat}
: > I suspect this may be related to 
[https://bugs.openjdk.java.net/browse/JDK-8209333] but i have no concrete 
evidence to back this up.
: > I'll post some details of my analysis in comments...
: 
: 
: 
: --
: This message was sent by Atlassian Jira
: (v8.3.4#803005)
: 

-Hoss

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-Solr-8.x-Solaris (64bit/jdk1.8.0) - Build # 462 - Still Unstable!

[Chris Hostetter]

Re-opened: https://issues.apache.org/jira/browse/SOLR-14054#

: Date: Wed, 18 Dec 2019 04:30:47 + (UTC)
: From: Policeman Jenkins Server 
: Reply-To: dev@lucene.apache.org
: To: bui...@lucene.apache.org
: Subject: [JENKINS] Lucene-Solr-8.x-Solaris (64bit/jdk1.8.0) - Build # 462 -
: Still Unstable!
: 
: Build: https://jenkins.thetaphi.de/job/Lucene-Solr-8.x-Solaris/462/
: Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseParallelGC
: 
: 36 tests failed.
: FAILED:  
junit.framework.TestSuite.org.apache.solr.handler.extraction.ExtractingRequestHandlerTest
: 
: Error Message:
: org/w3c/dom/ElementTraversal
: 
: Stack Trace:
: java.lang.NoClassDefFoundError: org/w3c/dom/ElementTraversal
:   at __randomizedtesting.SeedInfo.seed([29FFF67BA2F2C95E]:0)
:   at java.lang.ClassLoader.defineClass1(Native Method)
:   at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
:   at 
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
:   at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
:   at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
:   at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
:   at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
:   at java.security.AccessController.doPrivileged(Native Method)
:   at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
:   at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
:   at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
:   at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
:   at org.apache.xerces.parsers.AbstractDOMParser.startDocument(Unknown 
Source)
:   at org.apache.xerces.xinclude.XIncludeHandler.startDocument(Unknown 
Source)
:   at org.apache.xerces.impl.dtd.XMLDTDValidator.startDocument(Unknown 
Source)
:   at org.apache.xerces.impl.XMLDocumentScannerImpl.startEntity(Unknown 
Source)
:   at 
org.apache.xerces.impl.XMLVersionDetector.startDocumentParsing(Unknown Source)
:   at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
:   at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
:   at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
:   at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)
:   at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source)
:   at org.apache.solr.core.XmlConfigFile.(XmlConfigFile.java:140)
:   at org.apache.solr.core.XmlConfigFile.(XmlConfigFile.java:87)
:   at org.apache.solr.core.SolrConfig.(SolrConfig.java:173)
:   at org.apache.solr.core.SolrConfig.(SolrConfig.java:144)
:   at org.apache.solr.util.TestHarness.createConfig(TestHarness.java:89)
:   at org.apache.solr.SolrTestCaseJ4.createCore(SolrTestCaseJ4.java:812)
:   at org.apache.solr.SolrTestCaseJ4.initCore(SolrTestCaseJ4.java:805)
:   at org.apache.solr.SolrTestCaseJ4.initCore(SolrTestCaseJ4.java:666)
:   at 
org.apache.solr.handler.extraction.ExtractingRequestHandlerTest.beforeClass(ExtractingRequestHandlerTest.java:55)
:   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
:   at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
:   at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
:   at java.lang.reflect.Method.invoke(Method.java:498)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:878)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
:   at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
:   at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
:   at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
:   at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
:   at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
:   at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
:   at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
:   at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
:   at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
:   at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
:   at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
:   at 

Re: [JENKINS-EA] Lucene-Solr-master-Linux (64bit/jdk-14-ea+24) - Build # 25146 - Unstable!

[Chris Hostetter]

The last time DeleteShardTest ever had a failure was 2018-Feb ... anybody 
have any idea what might have changed to cause this?

It didn't reproduce, so it's possible it's a timing bug -- but 20 months 
straight w/o a failure strongly suggests something recent helps trigger 
whatever the underlying problem is.



: Subject: [JENKINS-EA] Lucene-Solr-master-Linux (64bit/jdk-14-ea+24) - Build #
: 25146 - Unstable!
: 
: Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/25146/
: Java: 64bit/jdk-14-ea+24 -XX:-UseCompressedOops -XX:+UseG1GC
: 
: 1 tests failed.
: FAILED:  org.apache.solr.cloud.DeleteShardTest.test
: 
: Error Message:
: Error from server at https://127.0.0.1:36547/solr: The slice: shard2 is 
currently active. Only non-active (or custom-hashed) slices can be deleted.
: 
: Stack Trace:
: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at https://127.0.0.1:36547/solr: The slice: shard2 is currently 
active. Only non-active (or custom-hashed) slices can be deleted.
:   at 
__randomizedtesting.SeedInfo.seed([565BFE4F53AC2C56:DE0FC195FD5041AE]:0)
:   at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:665)
:   at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:265)
:   at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
:   at 
org.apache.solr.client.solrj.impl.LBSolrClient.doRequest(LBSolrClient.java:368)
:   at 
org.apache.solr.client.solrj.impl.LBSolrClient.request(LBSolrClient.java:296)
:   at 
org.apache.solr.client.solrj.impl.BaseCloudSolrClient.sendRequest(BaseCloudSolrClient.java:1143)
:   at 
org.apache.solr.client.solrj.impl.BaseCloudSolrClient.requestWithRetryOnStaleState(BaseCloudSolrClient.java:906)
:   at 
org.apache.solr.client.solrj.impl.BaseCloudSolrClient.request(BaseCloudSolrClient.java:838)
:   at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:207)
:   at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:224)
:   at org.apache.solr.cloud.DeleteShardTest.test(DeleteShardTest.java:85)
:   at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
:   at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
:   at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
:   at java.base/java.lang.reflect.Method.invoke(Method.java:564)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
:   at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
:   at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
:   at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
:   at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
:   at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
:   at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
:   at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
:   at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
:   at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
:   at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
:   at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
:   at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
:   at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
:   at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
:   at 

Re: BadApple

[Chris Hostetter]

: I ran TestFstDirectAddressing.testDeDupeTails > 100 times with no
: failure, both on master and on branch_8x (on an Ubuntu w/a couple of
: different JDKs). Can you tell which branch/jvm/os is seeing the
: failures for that test? I couldn't tell from the attached report.

it's already been addressed - by you - in LUCENE-8920  (Erick's note was 
about it's overall success rate for the past 7 days)



: 
: On Mon, Nov 18, 2019 at 11:28 AM Erick Erickson  
wrote:
: >
: > Short form:
: >
: > Holding reasonably steady, except:
: >
: > PackageManagerCLITest.testPackeageManager is failing over 50% of the time.
: >
: > MoverReplicaHDFSTest.test failing over 40% of the time.
: >
: > TestFstDirectAddressing.testDeDupeTails is failing 18% of the time.
: >
: > Full output attached:
: >
: >
: > There were 154 unannotated tests that failed in Hoss' rollups. Ordered by 
the date I downloaded the rollup file, newest->oldest. See above for the dates 
the files were collected
: > These tests were NOT BadApple'd or AwaitsFix'd
: > All tests that failed 4 weeks running will be BadApple'd unless there are 
objections
: >
: > Failures in the last 4 reports..
: >Report   Pct runsfails   test
: >  0123   3.3  917 45  BasicAuthIntegrationTest.testBasicAuth
: >  0123   0.5  943  7  
DimensionalRoutedAliasUpdateProcessorTest.testCatTime
: >  0123   2.7  943 18  
DimensionalRoutedAliasUpdateProcessorTest.testTimeCat
: >  0123   8.3  961 91  
LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud
: >  0123  43.8  202 85  MoveReplicaHDFSTest.test
: >  0123   0.5  903  5  
ReindexCollectionTest.testSameTargetReindexing
: >  0123   4.8  104  5  ShardSplitTest.testSplitWithChaosMonkey
: >  0123   0.5  918  9  
SystemCollectionCompatTest.testBackCompat
: >  0123   4.8  941 52  
TestCloudSearcherWarming.testRepFactor1LeaderStartup
: >  0123   2.7  946 77  
TestModelManagerPersistence.testFilePersistence
: >  0123   2.2  940 70  
TestModelManagerPersistence.testWrapperModelPersistence
: >  0123   1.6  921 19  
TestSkipOverseerOperations.testSkipLeaderOperations
: >  0123   0.6  905 11  TestStressLiveNodes.testStress
: >  0123   1.1  651  9  TestTlogReplica.testKillTlogReplica
: >  0123   4.2  112 11  
TestXYMultiPolygonShapeQueries.testRandomBig
: >  Will BadApple all tests above this line except ones listed at 
the top**
: >
: >
: > -
: > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
: > For additional commands, e-mail: dev-h...@lucene.apache.org
: 
: -
: To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
: For additional commands, e-mail: dev-h...@lucene.apache.org
: 
: 

-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Do we want to pursue an LTS designation?

[Chris Hostetter]

: The term LTS is used loosely and we actually do not promise anything. What we 
say is
: 
: > 7.7.x   Version in the previous major release for bugfixes only (LTS)
...

: Being the one coining the LTS phrasing I am not opposed to changing it 
: to something else. But through the years we have used that term I have 
: not seen a lot of such requests.

I didn't realize you had put that up on the site -- it does seem very 
missleading to me as the "LT" in "LTS" is suppose to mean "Long Term" but 
we do not -- in practice or intent -- have any plans as a project to 
support "7.7.x" in particular any longer then we would any other arbitrary 
"latest" minor release.

Nor did we ever give users who initially installed 7.7.0 when it came out 
any reason to think it would be "supported" (and get bug fixes) for a 
"longer term" then a hypothetical 7.8.0 that might have come out a day 
after they installed some 7.7.x, and replace it's row on that page that 
you've labeled "LTS".

We may not have ever gotten questions about it, but that may just be 
because people have very specific assumptions about what it means, and 
never thought to ask questions to verify those assumptions.

I think, given how the project *actually* supports releases, I think we 
should remove references to "LTS" from that page, and leave the other 
description "Version in the previous major release for bugfixes only" as 
it stands -- or perhaps tighten it up even more: "Version in the previous 
major release that may get bug fixs"


-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Tracking down inconsistent failure in jenkins

[Chris Hostetter]

: Ok, I expected something like this. Why are some test timing related? Are 
there any informations about this.

multi-threaded code.  

In the extreme case, Test falures may be timing related due to buggy race 
conditions between multiple threads in "real code" (ie: very problematic 
for end users).

More typically what we see is race conditions between a thread in "real 
code" and a thread in "test code" (ie: a badly written test that doesn't 
harm any production user but is anoying to track down).

Timing factors betwen machines can also cause reproducibility problems in 
randomization logic -- the test framework tries to ensure every thread 
get's it's own consistent 'Random', but if there are multiple test threads 
that both interact with something (ie: a single SolrCore in a stress 
test), the order those threads are scheduled by the VM determines the 
order that the random values from each of those threads come into play -- 
ie: the order that 2 diff (random) updates from 2 diff threads hit solr, 
potentially triggering a bug in some situations (ie: maybe a certain 
ordering of updates).

other example of how  "timing" can cause reproducibility problems is in 
unpredictible ordering in lists which are then used to select things at 
random...

Example: maybe there is a bug with shutting down nodes that only manifests 
if the "leader" replica of a collection is being shutdown -- a 
(correctly written) test that might be garunteed that the 
seed DEADBEEF will cause it to always spin 
up exact 5 nodes and shutdown the 2nd node in the list -- but thread 
scheduling during collection creation may cause the 1st node in the list 
to _usually_ host the leader, but on rare occasion that 2nd node might 
host the leader because of which thread joined the election first.


...these are just some examples off the top of my head based on years of 
experience reading test logs ... and that's before we even talk about the 
possiblity/anoyance of actual race conditions/deadlocks in code, that 
might only trigger on slow machines (or fast machines) based on thread 
scheduling ... assuming we even have a test that causes those particular 
threads to try and run at the same time :)


-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [lucene-solr] branch branch_8x_tmp created (now f4eea9b)

[Chris Hostetter]

: Subject: [lucene-solr] branch branch_8x_tmp created (now f4eea9b)

WTF is this branch_8x_tmp !?!?!?!???

Since when is this ok?

-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

questionable behavior of DirectoryReader.getIndexCommit() on NRT Readers

[Chris Hostetter]

TL:DR: it seems like DirectoryReader.getIndexCommit() returns weird 
results when using a "reopened" reader off of uncommited IW changes.


Even though 2 diff readers will expose diff views of the index, they will 
claim to refer to the same IndexCommit.



Slightly Longer Version...

The test below, if added to TestIndexWriterReader, will pass reliably 
AFAICT -- but that seems really weird to me.  Note in particular the 
'nocommit' comments and the assertions that follow them -- these are very 
diff readers, exposing very diff views of the index, yet they claim to 
have the same "IndexCommit" & generation underpinning them, evne though 
some of the details of their IndexCommits differ


This seems weird, and I'm wondering if it's a bug, or an undespecified 
behavior (and we should beef up the docs to clarify what to expect), or if 
it represents some "feature" whose value i'm not understanding?



  public void testIndexCommitOfReaderAfterReopen() throws Exception {
final Directory dir = newDirectory();
final IndexWriterConfig iwc = new IndexWriterConfig(new 
MockAnalyzer(random()));
final IndexWriter w = new IndexWriter(dir, iwc);
try {
  final DirectoryReader r0 = DirectoryReader.open(w);
  try {
assertEquals(0, r0.numDocs());
final IndexCommit c0 = r0.getIndexCommit();
assertEquals(0L, c0.getGeneration());
  } finally {
r0.close();
  }

  w.addDocument(new Document());
  w.commit();

  final DirectoryReader r1 = DirectoryReader.open(w);
  try {
assertEquals(1, r1.numDocs());
final IndexCommit c1 = r1.getIndexCommit();
assertEquals(1L, c1.getGeneration());
assertEquals(1, c1.getSegmentCount());

w.addDocument(new Document());

final DirectoryReader r2 = DirectoryReader.openIfChanged(r1, w, true);
try {
  assertNotNull(r2);
  assertFalse(r1.equals(r2));
  assertEquals(2, r2.numDocs());

  // nocommit: Why do the assertions pass?
  //
  // nocommit: If the readers are not the same, and refer to different
  // nocommit: "point in time" views of the index, then shouldn't they
  // nocommit: also return different 'getIndexCommit()' results?
  //
  // nocommit: Should the "realtime" reader return 'null' since it's 
view
  // nocommit: of the index is not represented by a tangible commit?
  assertEquals(c1, r2.getIndexCommit());
  assertEquals(1L, r2.getIndexCommit().getGeneration());
  assertEquals(c1.getSegmentsFileName(), 
r2.getIndexCommit().getSegmentsFileName());
  // nocommit: particularly odd: even though the commits are ".equals()"
  // nocommit: and have the same generation & segments file,
  // nocommit: they have diff segment counts and files
  assertEquals(2, r2.getIndexCommit().getSegmentCount());
  assertTrue(! 
c1.getFileNames().equals(r2.getIndexCommit().getFileNames()));
} finally {
  r2.close();
}
  } finally {
r1.close();
  }
} finally {
  w.close();
  dir.close();
}
  }



Backstory: I'm digging into the Solr snapshot/backup code with an 
objective of fixing SOLR-13872, but before i move forward with changing 
anything I want to make sure I backfill the existing snapshot/backup test 
code with new test cases for all the code paths that seem underrepresented 
in tests so i don't risk introducing new bugs.  While trying to 
understand/test/document how Solr behaves if a user tries to request a 
snapshot/backup when using "softCommit" (at the solr level), I noticed 
this weird behavior of getting the same IndexCommit back from the 
reader even after we'd re-opened a Reader/Searcher w/o committing.




-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Rethinking how we publish the Solr Ref Guide

[Chris Hostetter]

: The redirection is wrong, if you remove "latest" from the urls with 8_1 in

The "redirection" rules appear to be working as designed -- but AFAIK they 
were never designed with any idea of having a "latest/" path.

the Latest URL has no is just the page name w/o a version number, not the 
page name prefixed with "latest/".

Specifically this is suppose to be the "latest" URL for the page in 
question...

https://lucene.apache.org/solr/guide/field-types-included-with-solr.html

: > I was trying to access
: > 
https://lucene.apache.org/solr/guide/latest/field-types-included-with-solr.html

...where did you find that link?

I'm pretty sure the place that linked to that URL is the place that has a 
mistake.



-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Solr Test Improvement PR From the Gradle Branch.

[Chris Hostetter]

: In the end I fixed a bunch more bs, pulled in a bunch of improvements and
: fixes around tests from the Starburst branch that had not come over yet,
: and have generally been wrapping my magic lasso around this test suite. A
: bunch of that includes stuff in the code that was causing test problems or
: slowness. I've put a lot of emphasis on moving actual slow stuff
: to @Nightly runs and fixing not actual slow stuff.

...

: I'd like to get it out of my gradle branch and into master before moving in
: Gradle, so I plan to put up a PR when I can in the near future and am
: hoping for some help with review.

So if i understand correctly: You've got a bunch of stuff on your gradle 
branch that is general test improvements/refactoring not dependent on or 
directly related to the ant->gradle migration at all; and as such you 
(wisely) want to review & commit it independently of the gradle migration.

correct?

I'm 100% on board helping to review & test those changes if you wnat to 
tease it out into it's own branch / PR ... just ping me once you have a 
jira so I don't miss it.


-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: The Lucene Solr Gradle Build Game plan

[Chris Hostetter]

: which should be a matter of project-local settings. Ideally, I'd like
: to see something like:
: 
: checkout/gradle-defaults.properties [versioned]
: checkout/gradle-local.properties [developer-tweaks, not versioned]
: 
: but short of declaring a custom "user home" for gradle (-g switch) I
: don't see how this can be achieved. Perhaps the initial checkout could

I gather from your comments (and from the link you mentioned in another 
question) that gradle does not have a direct equivilent of ant's...

  

...but some naive google searching suggests it's possible to do something 
similar using "apply from" to load in a *.gradle file using an arbitrary 
path, which could contain property declarations which would then evidently 
by overlayed on the existing groovy file? 
(maybe?)

Ex: this in our build.gradle file...

apply from: '$USER_HOME/lucene.properties.gradle'

...and this in my ~/lucene.properties.gradle file...

ext { 
test_jvms = 42
}

...would that work?


: > ("-Dtests.jvms=1" is important for figuring out if/how the execution 
: of one test might polute static variables in the JVM
: 
: I don't think it'll work even if you have one JVM unless Gradle test
: runner's order is deterministic... which I'm not sure it is, even with
: a single JVM that runs the tests.

FWIW: I'm not sure if the deterministic ordering is that important --
although you reminded me that being able to *review* the list of tests 
that ran in the same JVM when a failure happens is helpful...

(i don't know that i've ever tried to rerun that exact list of every test 
that ran in the same JVM in the same order as what was reported -- or even 
how i would if i wanted to other then with a really fucking complicated 
-Dtests.class=... param)

I'd guess that 90% of the time when i currently use "-Dtests.jvms=1" it's 
when running a single test class with -Dtests.dups=N to confirm & fix 
shitty use of statics in BeforeClass/AfterClass methods (typically 
problematic if/when the jenkins [repro] target does something like 
-Dtests.dups=5 -DtestcaseSomeTestThatFailed but defaults to using 
tests.jvms=3, so we see 2/5 suite level failures that look nothing like 
whatever the original failure was that jenkins was attempting to 
reproduce)

the 10% of the time -- where that's not the issue, and a failure in one 
test really was caused byt some previous test in the same JVM mucking up 
some static variable state -- eyeballing the list of tests that also ran 
in that JVM usually helps narrow down the list of suspects, and then i 
just try beasting those 2 (or 3) tests together with "-Dtests.jvms=1", not 
worrying about the order and just trusting that if i'm right eventually 
they'll be run in the problematic order.


-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Rethinking how we publish the Solr Ref Guide

[Chris Hostetter]

: > However Anshum does make a good point that users wouldn't know when 
: the pages have changed. I think it would be great to have a link on each 
: ref-guide page that shows the last modified date and links to the 
: history of that page in github

: Perhaps we could instead provide a single HTML page or HTML table as 
: part of or alongside each guide, showing all commits touching the guide 
: on that branch after the release. Could probably be baked in as part of 
: the release script. Using the release date or git hash for the release, 

Yeah, there are a lot of options we could pursue for generating a 
"changes" list as part of an automated build process -- but i would 
consider this idea a "nice to have" feature that shouldn't block moving 
forward.

Given 2 options, I would much rather:
  a) have the ability to quickly/easily "fix" mistakes/ommisions in 
"official X.y ref-guide" on our site and have it automatically republish, 
w/o it being immediately obvious to users that a page may have changed 
between yesterday and today.
  ... over ...
  b) *NOT* being able to re-publish at all just for the sake of users 
knowing that the (incorrect) content they are reading is consistent 
between yesterday and today.


-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Notes from the committers' meeting at Activate 2019

[Chris Hostetter]

: ASF has us legally covered, and from the foundation's side, GitHub 
: issues is equal to JIRA issues and GitHub PRs are equal to patches in 
: JIRA.

: > People that wish to continue using their Apache committer accounts to 
: commit code may continue doing so on gitbox.apache.org with their Apache 
: credentials. Nothing has changed in that respect.

: So the argument against TOS or personal M$ dislikes or principles won't hold 
here.
: 
: We can continue accepting JIRA issues, patches and commits to GitBox for 
: those who favor those tools for any reason. But we can equally well 
: embrace the entire GitHub tooling which was made available to us by ASF 

Ok -- everybody chill for a second.

I made a specific comment regarding github TOS/access in response to a 
*very* specific suggestion.

As a refresher, the specific suggestion i was responding to was this...

: > : Is there any reason at all that we need to hold on to JIRA? ASF allows 
: > : us to move all issue handling over to GH, I'd like us to consider such a 
: > : move.
: > 
: > In my opinion, migrating from JIRA to Github "issues" would be a terrible 
: > idea.

...that's it. *replacing* JIRA with github-issues is the specific idea i 
was saying was terrible.

Arguments that the code is still safe, and that committers who don't trust 
github can still push directly to gitbox w/o needing to accept 3rd party 
TOS; and that patches in github PRs are just as legally valid as patches 
in Jira are all fine -- but completely irrelevant to my comment.

Likewise: Arguments that people who don't agree to github TOS, or can't 
access github could still be contribute via JIRA make zero sense in the 
specific hypothetical scenerio i was replying to where JIRA no longer 
exists.


As i said before: if folks want to encourage and facilitate more direct 
integration and contributions & reviews via github -- using whatever weird 
ass github workflows or integrations or "hooks" or whatnot that github 
offers -- then cool, go for it, i'm all in favor of opening those doors 
(evne if i don't plan on using them much).

what i objected to was *closing* a door (again: specificly, migrating off 
of JIRA completely) that is currently open to anyone and saying it's not 
neccessary because we've open a new door that comes with a lot of 
restrictions and baggage.

: > I have no objections to the goal of "encouraging" and "facilitating" 
: > contributions via github from people already using github -- but making 
: > github the defacto (or *sole*) way to participate and contribute code 
: > means pressuring people into accepting the github TOS (not just 
: > now, but whatever those might be in the future) as well as making it 
: > virtually impossible for people to participate if they are in locations 
: > github has decided to block (ie: Iran, Syria, and Crimea ATM + whomever 
: > else the US decides to sanction down the road)
: > 
: > Opening up, or expanding, pathways for participation is one thing -- 
: > I'm all in favor of that (even if I personally can't stand those avenues).
: > 
: > But *closing* existing path ways that are currently entirely "open" and 
: > "free" to anyone that wants to participate w/o any limitations or TOS 
: > other then "Provide an ASF controled and owned website with an email 
: > address" ... that's just sad.


-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Rethinking how we publish the Solr Ref Guide

[Chris Hostetter]

First and foremost I should mention: I'm currently in favor of just about 
everything Cassandra has suggested here...

: So, I propose making some radical changes. My ideas here require a shift
: from thinking of the Guide as a release artifact like the binaries to
: thinking of it similar to how we treat javadocs. These ideas also allow us

I would actually go farther then that, and suggest that moving forward the 
"official ref-guide" for "Solr X.Y" (hosted on lucene.apache.org) 
automatically be updated anytime anyone pushes edits to brach_X_Y -- as 
opposed to our javadocs for X.Y.0 which *might* be rebuilt for formatting 
purposes (something we've done in the past) but no *code* (ie: "content") 
changes on branch_X_Y would be reflected ... those would be part of the 
"bug fix" release X.Y.1, which would have it's own javadocs.  But 
meanwhile, the ref-guide for X.Y could/would be updated with doc 
improvements even if there were no bug-fix releases from branch_X_Y.


: -- By ASF policy, release artifacts must be produced on a machine
: controlled by the committer. However, the point here is that the Ref Guide
: would no longer be a release artifact, so I think that gets us around that
: rule? If anyone sees this differently that would change things here a
: little bit.

FWIW: My understand from years ago was that the policy was unambiguious:
 1) a release vote is neccessary for anything that goes on dist.apache.org
 2) any "downloads" should come from dist.apache.org
...so "browseable html" docs on lucene.apache.org wouldn't require a 
VOTE, but if we want to keep providing a provide a big PDF or zip file of 
all the HTML that would require a vote -- *BUT* it seems like the rules 
are more ambiguious now, particularly regarding "documentation" downloads 
... ex: i know openoffice provides downloadable PDFs of their user guide 
from their wiki, pretty sure they don't vote on that.



: PS, As for the PDF, I believe there are mixed opinions about it. Some rely

As someone who has been a long time advocate of the PDF, and of treating 
it as an "official (VOTEed) release artifact" i wanted to toss out some 
historical context, and notes on how/why my own feelings have evolved.

Once upon a time, Solr had shit-all user documentation.  We had nothing 
but our (moin moin) wiki, which was a hodge-podge mess, an amalgmaation 
mix of docs written by developers as features were created, and "tips" 
pages written by users with thoughts on how to do things, and none of it 
was well organized and all of it was sprinkled with "this feature 
was added in version X.Y but changed defaults to FOO in version X.Z..."

When the lucidworks created the first few versions of the ref-guide using 
Confluence as a CMS, and donated it to the ASF, i (and others) really felt 
it was important that end users could see this new material as official, 
authoritative, and "specific" (to each version of Solr) ... we did *NOT* 
want people to start thinking of it as "just another wiki".  Since 
confluence didn't have an easy way to "branch" the entire space for 
each version (not w/o a lot of infra assistance) and *did* provide an easy 
way to publish the entire guide as a PDF, doing that and voting on the 
resulting PDF as a true "documentation release artifact" seemed like a 
good way to ensure we not only had version specific "snapshots" of the 
content, but gave these PDFs more "legtimacy" as being "official".

When we migrated to using asciidoctor, i (and others?) really felt like it 
was important to keep the continuity of having an "official PDF release 
artifact" since that was what our users were use to to ensure they were 
looking a the "correct" ref-guide version.  (With the old confluence CMS, 
the only "browseable" html view of the content corrisponded to the latest 
branch_YYY_x, with a handful of pages for trunk only features).  But of 
course: being able to branch the ref-guide source alongwith the code, and 
being able to build & host browseable HTML pages for all the content was a 
really nice value add.

The project, our documentation, and the communities views/experience with 
our documetation aren't the same as they were 6+ years ago.  As already 
mentioned by a few people: it seems like most users browse/read the 
(version specific) ref-guide hosted on lucene.apache.org.  The handful of 
users who really care about "offline" access to the content on their 
laptops can probably build the HTML site locally from source just as 
easily as they can downloda the PDF.

So while My 2013 self, and my 2015 self, and even my 2017 self would have 
been really adament about having an "official voted (PDF) release 
artifact" for the ref-guide ... my 2019 self realizes that the world has 
changed, and we're just making more work for ourselves -- at an 
opportunity cost of having an "official" (hosted) version of the ref-guide 
with more accurate "post release" doc fixes and more dynamic presentation 
features that just aren't 

Re: Notes from the committers' meeting at Activate 2019

[Chris Hostetter]

: Is there any reason at all that we need to hold on to JIRA? ASF allows 
: us to move all issue handling over to GH, I'd like us to consider such a 
: move.

In my opinion, migrating from JIRA to Github "issues" would be a terrible 
idea.

I have no objections to the goal of "encouraging" and "facilitating" 
contributions via github from people already using github -- but making 
github the defacto (or *sole*) way to participate and contribute code 
means pressuring people into accepting the github TOS (not just 
now, but whatever those might be in the future) as well as making it 
virtually impossible for people to participate if they are in locations 
github has decided to block (ie: Iran, Syria, and Crimea ATM + whomever 
else the US decides to sanction down the road)

Opening up, or expanding, pathways for participation is one thing -- 
I'm all in favor of that (even if I personally can't stand those avenues).

But *closing* existing path ways that are currently entirely "open" and 
"free" to anyone that wants to participate w/o any limitations or TOS 
other then "Provide an ASF controled and owned website with an email 
address" ... that's just sad.


-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: The Lucene Solr Gradle Build Game plan

[Chris Hostetter]

Some misc questions after playing around with gradle on this branch for a 
bit today in no particular order...

: tests_jvms=5
...
: test_jvms is controlled by us and defaults to the number of cores detected
: / 2.

If tests_jvms is a lucene/solr specific property, that needs to 
be in a users "multi-project" "~/.gradle" file ... shouldn't we name it 
namespace it with something like "lucene_solr_test_jvms" to make that 
clear reduce future confusion/collsion?  (as i anticipate this wo't be the 
last prop we may need)

How do we get "reproduce with" type output (by default) when a test fails? 

For that matter, how do we get *all* of the logging from failed tests to 
be written to stdout/stderr when a test fail?  ... this is pretty 
important for making jenkin's console log a good "one stop shop" for 
understanding everything that went wrong in a build.  

("--debug" and "--info" seem to do this, but they write a *TON* more 
gradle specific shit then "ant test" type logging use to produce by 
default, and don't care if the test passes or not ... which would make 
them way too excessively verbose for a jenkins build)

What's the plan as far as things like "ant beast", "-Dtests.dups", 
"-Dtests.iters" & "-Dtests.jvms" ? ... those types of options are all 
pretty critical for diagnosing and troubleshooting failures.  
("-Dtests.jvms=1" is important for figuring out if/how the execution of 
one test might polute static variables in the JVM that cause a failure in 
a subsequent class in the sane JVM)

is there a simple option to prevent gradle from using curses even though 
it detects it's being run in a tty?


-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: precommit fail or is it me?

[Chris Hostetter]

My guess is you somehow have a very old jar (or sha1 file that is leading 
it to look for a jar you don't have) for an outdated version of jetty -- 
we are certainly not using jetty 9.4.14.v20181114 in master or branch_8x

what does `find -name \*jetty-continuation\*` report on your system?

does `ant clean clean-jars` help?

what does `git clean --dry-run -dx` say after you try to run ant clean 
clean-jars?

(it won't delet anything with --dry-run, but it might tell you if you have 
unexpected stuff)

: Date: Fri, 6 Sep 2019 09:53:30 -0400
: From: Michael Sokolov 
: Reply-To: dev@lucene.apache.org
: To: java-...@lucene.apache.org
: Subject: precommit fail or is it me?
: 
: Is anybody else seeing this error:
: 
: ...workspace/lucene/lucene_baseline/build.xml:117: The following error
: occurred while executing this line:
: .../workspace/lucene/lucene_baseline/lucene/build.xml:90: The
: following error occurred while executing this line:
: .../workspace/lucene/lucene_baseline/lucene/tools/custom-tasks.xml:62:
: JAR resource does not exist:
: replicator/lib/jetty-continuation-9.4.14.v20181114.jar
: 
: Should I be using gradle instead of ant precommit??
: 
: -
: To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
: For additional commands, e-mail: dev-h...@lucene.apache.org
: 
: 

-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Failing tests

[Chris Hostetter]

FWIW: it looks like Uwe already created a new Jenkins JIRA for this, and 
there is already a new PR to (try and) fix the new problem.

I created a github issue for my "jenkins-reports" side project (which is 
what's consuming these RSS feeds and generating the reports) just so there 
could be a single plass to track everything related to this (didn't make 
sense to create a SOLR jira for this) ...

https://github.com/hossman/jenkins-reports/issues/1


: Date: Sat, 24 Aug 2019 15:01:59 -
: From: Uwe Schindler 
: Reply-To: dev@lucene.apache.org
: To: dev@lucene.apache.org
: Subject: Re: Failing tests
: 
: Hi,
: 
: Jenkins was updated, the bug seems to be (partially) fixed, but the URL is
: no longer absolute. Not sure why.
: 
: I am on vacation, so could anybody check this and maybe reopen the jenkins
: issue, if there is still a regression?
: 
: Uwe
: 
: > : Just for yucks, I grepped the e-mails I’ve been sending out for the
: > : number of failing tests in the most recent 4 of Hoss’s rollups, see
: > : below.
: >
: > my reports aren't comprehensive for the past few weeks because of a bug in
: > Uwe's jenkins box that broke data collection...
: >
: > 
http://mail-archives.apache.org/mod_mbox/lucene-dev/201908.mbox/%3c010b01d551d0$6ae88b50$40b9a1f0$@thetaphi.de%3e
: >
: >
: > :
: > : The drop in the last few weeks is dramatic, hope it’s a trend…..
: > :
: > : e-mail-2018-06-11.txt:  989
: > : e-mail-2018-06-18.txt:  689
: > : e-mail-2018-06-25.txt:  555
: > : e-mail-2018-07-02.txt:  723
: > : e-mail-2018-07-09.txt:  793
: > : e-mail-2018-07-16.txt:  809
: > : e-mail-2018-07-23.txt:  953
: > : e-mail-2018-07-30.txt:  945
: > : e-mail-2018-08-06.txt:  830
: > : e-mail-2018-08-14.txt:  853
: > : e-mail-2018-08-20.txt:  547
: > : e-mail-2018-08-27.txt:  571
: > : e-mail-2018-09-03.txt:  512
: > : e-mail-2018-09-10.txt:  605
: > : e-mail-2018-09-18.txt:  624
: > : e-mail-2018-10-08.txt:  816
: > : e-mail-2018-12-24.txt: 1050
: > : e-mail-2019-01-08.txt:  655
: > : e-mail-2019-01-15.txt:  421
: > : e-mail-2019-02-12.txt:  347
: > : e-mail-2019-02-18.txt:  341
: > : e-mail-2019-03-04.txt:  279
: > : e-mail-2019-03-11.txt:  301
: > : e-mail-2019-03-18.txt:  275
: > : e-mail-2019-03-25.txt:  279
: > : e-mail-2019-04-01.txt:  288
: > : e-mail-2019-04-08.txt:  288
: > : e-mail-2019-04-15.txt:  259
: > : e-mail-2019-04-22.txt:  260
: > : e-mail-2019-05-06.txt:  238
: > : e-mail-2019-05-20.txt:  222
: > : e-mail-2019-06-03.txt:  199
: > : e-mail-2019-06-10.txt:  258
: > : e-mail-2019-06-17.txt:  530
: > : e-mail-2019-06-24.txt:  543
: > : e-mail-2019-07-01.txt:  585
: > : e-mail-2019-07-29.txt:  338
: > : e-mail-2019-08-05.txt:  252
: > : e-mail-2019-08-12.txt:  182
: > : e-mail-2019-08-19.txt:   80
: > : -
: > : To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
: > : For additional commands, e-mail: dev-h...@lucene.apache.org
: > :
: > :
: >
: > -Hoss
: > http://www.lucidworks.com/
: >
: > -
: > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
: > For additional commands, e-mail: dev-h...@lucene.apache.org
: 
: 
: -
: To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
: For additional commands, e-mail: dev-h...@lucene.apache.org
: 
: 

-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Failing tests

[Chris Hostetter]

: Just for yucks, I grepped the e-mails I’ve been sending out for the 
: number of failing tests in the most recent 4 of Hoss’s rollups, see 
: below.

my reports aren't comprehensive for the past few weeks because of a bug in 
Uwe's jenkins box that broke data collection...

http://mail-archives.apache.org/mod_mbox/lucene-dev/201908.mbox/%3c010b01d551d0$6ae88b50$40b9a1f0$@thetaphi.de%3e


: 
: The drop in the last few weeks is dramatic, hope it’s a trend…..
: 
: e-mail-2018-06-11.txt:  989 
: e-mail-2018-06-18.txt:  689 
: e-mail-2018-06-25.txt:  555 
: e-mail-2018-07-02.txt:  723 
: e-mail-2018-07-09.txt:  793 
: e-mail-2018-07-16.txt:  809 
: e-mail-2018-07-23.txt:  953 
: e-mail-2018-07-30.txt:  945 
: e-mail-2018-08-06.txt:  830 
: e-mail-2018-08-14.txt:  853 
: e-mail-2018-08-20.txt:  547 
: e-mail-2018-08-27.txt:  571 
: e-mail-2018-09-03.txt:  512 
: e-mail-2018-09-10.txt:  605 
: e-mail-2018-09-18.txt:  624 
: e-mail-2018-10-08.txt:  816 
: e-mail-2018-12-24.txt: 1050 
: e-mail-2019-01-08.txt:  655 
: e-mail-2019-01-15.txt:  421 
: e-mail-2019-02-12.txt:  347 
: e-mail-2019-02-18.txt:  341 
: e-mail-2019-03-04.txt:  279 
: e-mail-2019-03-11.txt:  301 
: e-mail-2019-03-18.txt:  275 
: e-mail-2019-03-25.txt:  279 
: e-mail-2019-04-01.txt:  288 
: e-mail-2019-04-08.txt:  288 
: e-mail-2019-04-15.txt:  259 
: e-mail-2019-04-22.txt:  260 
: e-mail-2019-05-06.txt:  238 
: e-mail-2019-05-20.txt:  222 
: e-mail-2019-06-03.txt:  199 
: e-mail-2019-06-10.txt:  258 
: e-mail-2019-06-17.txt:  530 
: e-mail-2019-06-24.txt:  543 
: e-mail-2019-07-01.txt:  585 
: e-mail-2019-07-29.txt:  338 
: e-mail-2019-08-05.txt:  252 
: e-mail-2019-08-12.txt:  182 
: e-mail-2019-08-19.txt:   80
: -
: To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
: For additional commands, e-mail: dev-h...@lucene.apache.org
: 
: 

-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-Solr-8.x-MacOSX (64bit/jdk-11.0.3) - Build # 271 - Still Failing!

[Chris Hostetter]

https://issues.apache.org/jira/browse/SOLR-13688

: Date: Tue, 13 Aug 2019 19:48:25 + (UTC)
: From: Policeman Jenkins Server 
: Reply-To: dev@lucene.apache.org
: To: dev@lucene.apache.org
: Subject: [JENKINS] Lucene-Solr-8.x-MacOSX (64bit/jdk-11.0.3) - Build # 271 -
: Still Failing!
: 
: Build: https://jenkins.thetaphi.de/job/Lucene-Solr-8.x-MacOSX/271/
: Java: 64bit/jdk-11.0.3 -XX:-UseCompressedOops -XX:+UseSerialGC
: 
: All tests passed
: 
: Build Log:
: [...truncated 12279 lines...]
: [javac] Compiling 1284 source files to 
/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/classes/java
: [javac] 
/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/src/java/org/apache/solr/util/ExportTool.java:312:
 error: cannot infer type arguments for BiConsumer
: [javac] private BiConsumer bic= new BiConsumer<>() {
: [javac]   ^
: [javac]   reason: '<>' with anonymous inner classes is not supported in 
-source 8
: [javac] (use -source 9 or higher to enable '<>' with anonymous inner 
classes)
: [javac]   where T,U are type-variables:
: [javac] T extends Object declared in interface BiConsumer
: [javac] U extends Object declared in interface BiConsumer
: [javac] Note: Some input files use or override a deprecated API.
: [javac] Note: Recompile with -Xlint:deprecation for details.
: [javac] Note: Some input files use unchecked or unsafe operations.
: [javac] Note: Recompile with -Xlint:unchecked for details.
: [javac] 1 error
: 
: BUILD FAILED
: /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/build.xml:634: The following 
error occurred while executing this line:
: /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/build.xml:578: The following 
error occurred while executing this line:
: /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/build.xml:59: The following 
error occurred while executing this line:
: /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build.xml:231: The 
following error occurred while executing this line:
: /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/common-build.xml:558: 
The following error occurred while executing this line:
: /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/common-build.xml:506: 
The following error occurred while executing this line:
: /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/common-build.xml:401: 
The following error occurred while executing this line:
: /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/lucene/common-build.xml:558: 
The following error occurred while executing this line:
: /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/lucene/common-build.xml:2056: 
Compile failed; see the compiler error output for details.
: 
: Total time: 30 minutes 40 seconds
: Build step 'Invoke Ant' marked build as failure
: Archiving artifacts
: Setting 
ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
: [WARNINGS] Skipping publisher since build result is FAILURE
: Recording test results
: Setting 
ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
: Email was triggered for: Failure - Any
: Sending email for trigger: Failure - Any
: Setting 
ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
: Setting 
ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
: Setting 
ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
: Setting 
ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
: Setting 
ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
: Setting 
ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
: 

-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

thetaphi jenkins RSS feed broken? (not showing up in fucit reports since July 24th)

[Chris Hostetter]

Uwe: I just realized that my jenkins reports haven't mentioned any 
failures from your jenkins box all week -- and that's because aparently 
even though the the RSS feed is up to date, and lists recent jobs, the 
URLs are all "wrong" and so the script can't get the test results & 
logs.

http://fucit.org/solr-jenkins-reports/


Note the "link" for this single entry currently in your RSS feed...

https://jenkins.thetaphi.de/view/Lucene-Solr/rssAll

 Lucene-Solr-master-Windows #8073 (9 more tests are failing (total 
16))
 https://jenkins.thetaphi.de/"/>
 tag:hudson.dev.java.net,2019:Lucene-Solr-master-Windows:8073
 
 
 Java: 64bit/jdk-11.0.3 -XX:+UseCompressedOops 
-XX:+UseConcMarkSweepGC


...every "entry" in your jenkin's feed has that exact same idential link 
href -- just for the base URL of your server, w/o the job specifics path.


Also note, if it helps track down the problem, that the published & 
updated dates are also blank (which i think my RSS crawler could handle 
by assuming 'now' if the 'link' was valid)


The problem seems to have started on July 24th (that's the last time my 
scripts reported seeing a "new" jenkins job in our feed -- with a valid 
'link')



For contrast, compare to what entries from the apache jenkins RSS feed 
look like ...


https://builds.apache.org/view/L/view/Lucene/rssAll

 Lucene-Solr-Tests-master #3516 (broken since build #3515)
 https://builds.apache.org/view/L/view/Lucene/job/Lucene-Solr-Tests-master/3516/"/>
 tag:hudson.dev.java.net,2019:Lucene-Solr-Tests-master:3516
 2019-08-07T14:12:25Z
 2019-08-07T14:12:25Z




-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-Solr-Tests-master - Build # 3453 - Failure

[Chris Hostetter]

This failure is caused by a JDK HashMap AssertionError, which 
reproduces fairly easily for me using java11, but not with java12.

I *think* it's the same bug as JDK-8205399 -- first identified in java10, 
and fixed in java12...

https://bugs.openjdk.java.net/browse/JDK-8205399

I filed a SOLR issue to try and capture for posterity how this bug can 
affect Solr users, but it's not clear to me how big the impact is (does 
this cause data corruption in the HashMap, or just loss of performance in 
the balance of the RB Tree?) ... and either way there doesn't seem to be 
much we can do about it?

https://issues.apache.org/jira/browse/SOLR-13653

What really confuses me, is that the "fix" for JDK-8205399 was identified 
and committed to the OpenJDK repo 2018-08-10, but doesn't seem to have 
been backported for inclusion in either 11.0.2 or 11.0.3.

(/cc rory in case he can provide any insights)




: Date: Thu, 25 Jul 2019 04:42:40 + (UTC)
: From: Apache Jenkins Server 
: Reply-To: dev@lucene.apache.org
: To: dev@lucene.apache.org
: Subject: [JENKINS] Lucene-Solr-Tests-master - Build # 3453 - Failure
: 
: Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/3453/
: 
: 1 tests failed.
: FAILED:  org.apache.solr.search.facet.TestCloudJSONFacetSKG.testRandom
: 
: Error Message:
: Error from server at 
http://127.0.0.1:42227/solr/org.apache.solr.search.facet.TestCloudJSONFacetSKG_collection:
 Expected mime type application/octet-stream but got text/html.   
 
Error 500 Server Error  HTTP ERROR 500 
Problem accessing 
/solr/org.apache.solr.search.facet.TestCloudJSONFacetSKG_collection/select. 
Reason: Server ErrorCaused 
by:java.lang.AssertionError  at 
java.base/java.util.HashMap$TreeNode.moveRootToFront(HashMap.java:1896)  at 
java.base/java.util.HashMap$TreeNode.putTreeVal(HashMap.java:2061)  at 
java.base/java.util.HashMap.putVal(HashMap.java:633)  at 
java.base/java.util.HashMap.put(HashMap.java:607)  at 
org.apache.solr.search.LRUCache.put(LRUCache.java:201)  at 
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1449)
  at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:
 568)  at 
org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryComponent.java:1484)
  at 
org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:398)
  at 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:305)
  at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
  at org.apache.solr.core.SolrCore.execute(SolrCore.java:2581)  at 
org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:780)  at 
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:566)  at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:423)
  at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:350)
  at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
  at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:165)
  at org.eclipse.jetty.servlet.ServletHandler$Cac
 hedChain.doFilter(ServletHandler.java:1610)  at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)  at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
  at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1711)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
  at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1347)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)  
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1678)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
  at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1249)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)  
at org.eclipse.jetty.serv
 er.handler.gzip.GzipHandler.handle(GzipHandler.java:753)  at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) 
 at org.eclipse.jetty.server.Server.handle(Server.java:505)  at 
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370)  at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267)  at 
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
  at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)  at 
org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)  at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:781)
  at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:917)
  at 

Re: [JENKINS] Lucene-Solr-NightlyTests-8.x - Build # 158 - Still Unstable

[Chris Hostetter]

: The BasicAuthIntegrationTest failures seem to be due to SOLR-13545
: (FYI, Mikhail).

Ishan: was that just a hypothosis based on the issue summary realating to 
a type of ContentStream, or do you have some specific testing that 
suggests the failure rate increased as a result of Mikhail's commits?

Because all the failing seeds i've seen recently reproduce reliably 100% 
of the time, and using git bisect to vet some of them identifies 
SOLR-13637 as the root cause (NOT SOLR-13545)

But now i'm wondering if there are specific seeds you've seen that fail 
differently then the once i've investigated? ... your comment was somewhat 
vague, particularly since *this* build (Lucene-Solr-NightlyTests-8.x - 
Build # 158) is not a jenkins run where BasicAuthIntegrationTest 
actually failed)


: On Tue, Jul 23, 2019 at 12:15 PM Noble Paul  wrote:
: > 
: > I'm looking into the failures
: >
: > On Tue, Jul 23, 2019 at 9:21 AM Apache Jenkins Server
: >  wrote:
: > >
: > > Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-8.x/158/
: > >
: > > 4 tests failed.
: > > FAILED:  
org.apache.solr.cloud.cdcr.CdcrReplicationHandlerTest.testReplicationWithBufferedUpdates
: > >
: > > Error Message:
: > > Timeout while trying to assert number of documents @ source_collection
: > >
: > > Stack Trace:
: > > java.lang.AssertionError: Timeout while trying to assert number of 
documents @ source_collection
: > > at 
__randomizedtesting.SeedInfo.seed([4C51580235DFBD67:9F58081C704C21F0]:0)
: > > at 
org.apache.solr.cloud.cdcr.BaseCdcrDistributedZkTest.assertNumDocs(BaseCdcrDistributedZkTest.java:277)
: > > at 
org.apache.solr.cloud.cdcr.CdcrReplicationHandlerTest.testReplicationWithBufferedUpdates(CdcrReplicationHandlerTest.java:233)
: > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
: > > at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
: > > at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
: > > at java.lang.reflect.Method.invoke(Method.java:498)
: > > at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
: > > at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
: > > at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
: > > at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
: > > at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1082)
: > > at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:1054)
: > > at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
: > > at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
: > > at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
: > > at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
: > > at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
: > > at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
: > > at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: > > at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
: > > at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
: > > at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
: > > at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
: > > at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
: > > at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
: > > at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
: > > at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: > > at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
: > > at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
: > > at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: > > at 

Re: [JENKINS] Lucene-Solr-SmokeRelease-8.x - Build # 150 - Still Failing

[Chris Hostetter]

Ugh, sorry steve -- thatnks for being on top of this.

(i should have realized there was probably an open Jira for this already, 
but i didn't remember seeing any replies to other recent jenkins failures 
... probably missed it over the July4 weekend)


: Date: Tue, 16 Jul 2019 13:14:33 -0400
: From: Steve Rowe 
: Reply-To: dev@lucene.apache.org
: To: dev@lucene.apache.org
: Subject: Re: [JENKINS] Lucene-Solr-SmokeRelease-8.x - Build # 150 - Still
: Failing
: 
: https://issues.apache.org/jira/browse/INFRA-18701 
<https://issues.apache.org/jira/browse/INFRA-18701> <- report of problem to 
Infra, who don't seem interested (follow linked issue to find other projects 
reporting the same problem)
: 
: related: https://issues.apache.org/jira/browse/INFRA-18505 
<https://issues.apache.org/jira/browse/INFRA-18505> <- lucene1 VM 
reconfiguration
: 
: --
: Steve
: 
: > On Jul 16, 2019, at 1:11 PM, Chris Hostetter  
wrote:
: > 
: > 
: > Same problem as the Lucene-Solr-SmokeRelease-8.2 jenkins job ... is the 
: > problem that java9 isn't installed on the apache jenkins VMs?
: > 
: > : prepare-release-no-sign:
: > : [mkdir] Created dir: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-8.x/lucene/build/smokeTestRelease/dist
: > :  [copy] Copying 502 files to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-8.x/lucene/build/smokeTestRelease/dist/lucene
: > :  [copy] Copying 226 files to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-8.x/lucene/build/smokeTestRelease/dist/solr
: > :[smoker] Java 1.8 JAVA_HOME=/home/jenkins/tools/java/latest1.8
: > :[smoker] Java 9 JAVA_HOME=/home/jenkins/tools/java/latest1.9
: > :[smoker] Traceback (most recent call last):
: > :[smoker]   File 
"/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-8.x/dev-tools/scripts/smokeTestRelease.py",
 line 1485, in 
: > :[smoker] main()
: > :[smoker]   File 
"/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-8.x/dev-tools/scripts/smokeTestRelease.py",
 line 1403, in main
: > :[smoker] c = parse_config()
: > :[smoker]   File 
"/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-8.x/dev-tools/scripts/smokeTestRelease.py",
 line 1266, in parse_config
: > :[smoker] c.java = make_java_config(parser, c.test_java9)
: > :[smoker]   File 
"/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-8.x/dev-tools/scripts/smokeTestRelease.py",
 line 1213, in make_java_config
: > :[smoker] run_java9 = _make_runner(java9_home, '9')
: > :[smoker]   File 
"/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-8.x/dev-tools/scripts/smokeTestRelease.py",
 line 1201, in _make_runner
: > :[smoker] shell=True, stderr=subprocess.STDOUT).decode('utf-8')
: > :[smoker]   File "/usr/lib/python3.4/subprocess.py", line 620, in 
check_output
: > :[smoker] raise CalledProcessError(retcode, process.args, 
output=output)
: > :[smoker] subprocess.CalledProcessError: Command 'export 
JAVA_HOME="/home/jenkins/tools/java/latest1.9" 
PATH="/home/jenkins/tools/java/latest1.9/bin:$PATH" 
JAVACMD="/home/jenkins/tools/java/latest1.9/bin/java"; java -version' returned 
non-zero exit status 127
: > : 
: > : BUILD FAILED
: > : 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-8.x/build.xml:462: 
exec returned: 1
: > : 
: > : Total time: 11 minutes 30 seconds
: > : Build step 'Invoke Ant' marked build as failure
: > : Setting JDK_1_9_LATEST__HOME=/home/jenkins/tools/java/latest1.9
: > : Setting JDK_1_9_LATEST__HOME=/home/jenkins/tools/java/latest1.9
: > : Email was triggered for: Failure - Any
: > : Sending email for trigger: Failure - Any
: > : Setting JDK_1_9_LATEST__HOME=/home/jenkins/tools/java/latest1.9
: > : Setting JDK_1_9_LATEST__HOME=/home/jenkins/tools/java/latest1.9
: > : Setting JDK_1_9_LATEST__HOME=/home/jenkins/tools/java/latest1.9
: > : Setting JDK_1_9_LATEST__HOME=/home/jenkins/tools/java/latest1.9
: > : 
: > 
: > -Hoss
: > http://www.lucidworks.com/ <http://www.lucidworks.com/>
: > 
: > -
: > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org 
<mailto:dev-unsubscr...@lucene.apache.org>
: > For additional commands, e-mail: dev-h...@lucene.apache.org 
<mailto:dev-h...@lucene.apache.org>
: 

-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-Solr-SmokeRelease-8.x - Build # 150 - Still Failing

[Chris Hostetter]

Same problem as the Lucene-Solr-SmokeRelease-8.2 jenkins job ... is the 
problem that java9 isn't installed on the apache jenkins VMs?

: prepare-release-no-sign:
: [mkdir] Created dir: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-8.x/lucene/build/smokeTestRelease/dist
:  [copy] Copying 502 files to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-8.x/lucene/build/smokeTestRelease/dist/lucene
:  [copy] Copying 226 files to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-8.x/lucene/build/smokeTestRelease/dist/solr
:[smoker] Java 1.8 JAVA_HOME=/home/jenkins/tools/java/latest1.8
:[smoker] Java 9 JAVA_HOME=/home/jenkins/tools/java/latest1.9
:[smoker] Traceback (most recent call last):
:[smoker]   File 
"/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-8.x/dev-tools/scripts/smokeTestRelease.py",
 line 1485, in 
:[smoker] main()
:[smoker]   File 
"/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-8.x/dev-tools/scripts/smokeTestRelease.py",
 line 1403, in main
:[smoker] c = parse_config()
:[smoker]   File 
"/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-8.x/dev-tools/scripts/smokeTestRelease.py",
 line 1266, in parse_config
:[smoker] c.java = make_java_config(parser, c.test_java9)
:[smoker]   File 
"/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-8.x/dev-tools/scripts/smokeTestRelease.py",
 line 1213, in make_java_config
:[smoker] run_java9 = _make_runner(java9_home, '9')
:[smoker]   File 
"/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-8.x/dev-tools/scripts/smokeTestRelease.py",
 line 1201, in _make_runner
:[smoker] shell=True, stderr=subprocess.STDOUT).decode('utf-8')
:[smoker]   File "/usr/lib/python3.4/subprocess.py", line 620, in 
check_output
:[smoker] raise CalledProcessError(retcode, process.args, output=output)
:[smoker] subprocess.CalledProcessError: Command 'export 
JAVA_HOME="/home/jenkins/tools/java/latest1.9" 
PATH="/home/jenkins/tools/java/latest1.9/bin:$PATH" 
JAVACMD="/home/jenkins/tools/java/latest1.9/bin/java"; java -version' returned 
non-zero exit status 127
: 
: BUILD FAILED
: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-8.x/build.xml:462: 
exec returned: 1
: 
: Total time: 11 minutes 30 seconds
: Build step 'Invoke Ant' marked build as failure
: Setting JDK_1_9_LATEST__HOME=/home/jenkins/tools/java/latest1.9
: Setting JDK_1_9_LATEST__HOME=/home/jenkins/tools/java/latest1.9
: Email was triggered for: Failure - Any
: Sending email for trigger: Failure - Any
: Setting JDK_1_9_LATEST__HOME=/home/jenkins/tools/java/latest1.9
: Setting JDK_1_9_LATEST__HOME=/home/jenkins/tools/java/latest1.9
: Setting JDK_1_9_LATEST__HOME=/home/jenkins/tools/java/latest1.9
: Setting JDK_1_9_LATEST__HOME=/home/jenkins/tools/java/latest1.9
: 

-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-Solr-SmokeRelease-8.2 - Build # 5 - Still Failing

[Chris Hostetter]

Uh ... what?

:[smoker] subprocess.CalledProcessError: Command 'export 
JAVA_HOME="/home/jenkins/tools/java/latest1.9" 
PATH="/home/jenkins/tools/java/latest1.9/bin:$PATH" 
JAVACMD="/home/jenkins/tools/java/latest1.9/bin/java"; java -version' returned 
non-zero exit status 127

 

: prepare-release-no-sign:
: [mkdir] Created dir: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-8.2/lucene/build/smokeTestRelease/dist
:  [copy] Copying 502 files to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-8.2/lucene/build/smokeTestRelease/dist/lucene
:  [copy] Copying 226 files to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-8.2/lucene/build/smokeTestRelease/dist/solr
:[smoker] Java 1.8 JAVA_HOME=/home/jenkins/tools/java/latest1.8
:[smoker] Java 9 JAVA_HOME=/home/jenkins/tools/java/latest1.9
:[smoker] Traceback (most recent call last):
:[smoker]   File 
"/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-8.2/dev-tools/scripts/smokeTestRelease.py",
 line 1485, in 
:[smoker] main()
:[smoker]   File 
"/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-8.2/dev-tools/scripts/smokeTestRelease.py",
 line 1403, in main
:[smoker] c = parse_config()
:[smoker]   File 
"/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-8.2/dev-tools/scripts/smokeTestRelease.py",
 line 1266, in parse_config
:[smoker] c.java = make_java_config(parser, c.test_java9)
:[smoker]   File 
"/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-8.2/dev-tools/scripts/smokeTestRelease.py",
 line 1213, in make_java_config
:[smoker] run_java9 = _make_runner(java9_home, '9')
:[smoker]   File 
"/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-8.2/dev-tools/scripts/smokeTestRelease.py",
 line 1201, in _make_runner
:[smoker] shell=True, stderr=subprocess.STDOUT).decode('utf-8')
:[smoker]   File "/usr/lib/python3.4/subprocess.py", line 620, in 
check_output
:[smoker] raise CalledProcessError(retcode, process.args, output=output)
:[smoker] subprocess.CalledProcessError: Command 'export 
JAVA_HOME="/home/jenkins/tools/java/latest1.9" 
PATH="/home/jenkins/tools/java/latest1.9/bin:$PATH" 
JAVACMD="/home/jenkins/tools/java/latest1.9/bin/java"; java -version' returned 
non-zero exit status 127
: 
: BUILD FAILED
: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-8.2/build.xml:462: 
exec returned: 1
: 
: Total time: 12 minutes 8 seconds
: Build step 'Invoke Ant' marked build as failure
: Setting JDK_1_9_LATEST__HOME=/home/jenkins/tools/java/latest1.9
: Setting JDK_1_9_LATEST__HOME=/home/jenkins/tools/java/latest1.9
: Email was triggered for: Failure - Any
: Sending email for trigger: Failure - Any
: Setting JDK_1_9_LATEST__HOME=/home/jenkins/tools/java/latest1.9
: Setting JDK_1_9_LATEST__HOME=/home/jenkins/tools/java/latest1.9
: Setting JDK_1_9_LATEST__HOME=/home/jenkins/tools/java/latest1.9
: Setting JDK_1_9_LATEST__HOME=/home/jenkins/tools/java/latest1.9
: 

-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

RE: Need to upgrade jenkins jdk-11 jobs >= 11.0.3 to fix JVM SSL bugs

[Chris Hostetter]

: no changes in the VM configurations (same number of cores,  everything same). 
But one change:

hmmm, ok .. jus a shot in the dark.  thanks for confirming.

: So it was updated to the Windows 10 Feature Update 1903… an 22nd of 
: June. I have no idea what this change may have caused in our tests; but 
: if there is something different, it would also affect Windows Server 
: users upgrading to latest Windows Server versions.

yeah ... i'm really not sure how anything OS level could be causing this 
... even my "CPU contention" guess doesn't _really_ make sense to me, 
since earlier points in the test are basically doing the exact same thing, 
and should need basically the same number of threads ... yet when the test 
fails it also fails at this exact spot.

weird.


-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

RE: Need to upgrade jenkins jdk-11 jobs >= 11.0.3 to fix JVM SSL bugs

[Chris Hostetter]

Uwe: when you upgraded the JVMs, were there by any chance other 
(potentially inadvertent) changes to the VMs -- particularly the Windows 
VM?

Since June 22nd, we've see ReplicationFactorTest fail in _almost_ every 
Windows build from your server, regardless of JVM version used, on 
both master and branch_8x -- and when it fails, the failure repro 
checks at the end of the build "succeed" in reproducing the failure 
anywhere from 1-5 times.  All of the failures occur at the exact same line 
number of the test.

(Prior to June 22nd, this test only failed a total of 5 times in 2019, 
across all jenkins servers/OSes, etc... and at a quick glance, rarely in 
this specific spot)

Based on my review of a handful of the logs from the past week, it looks 
like the failures *may* be caused by CPU starvation -- but even that's a 
guess since similar situations are also tested earlier in the same test -- 
and it never seems to fail during them (as it simulates 3 servers with one 
or 2 of the replicas partitioned off via a closed proxy, one update thread 
on the leader is continuously trying to reconnect, while another leader 
thread is sending updates to the remaining "live" replica -- but that live 
replica doesn't seem to "see" the request until 30 seconds later, after 
the first update thread appears to have given up on the "down" replica)

in any case -- i'm wondering if perhaps the number of "virtual 
CPUs" or some other VM realted setting might have changed on your Windows 
test instance.



: Date: Sat, 22 Jun 2019 18:37:42 +0200
: From: Uwe Schindler 
: To: dev@lucene.apache.org, Chris Hostetter 
: Subject: RE: Need to upgrade jenkins jdk-11 jobs >= 11.0.3 to fix JVM SSL bugs
: 
: Hi Hoss,
: 
:  
: 
: sorry for the delay, I updated JDK on Policeman Jenkins. But there are some 
things to mention:
: 
:  
: 
: * JDK 11.0.3 is *not* available free of charge from Oracle nor they 
supply OpenJDK convenience builds. Because of this we have to use JDK 11.0.3 
from AdoptOpenJDK (Hotspot variant). I have installed those builds on Linux, 
Windows, MacOSX
: * JDK 12.0.1 was also installed on Linux, Windows, MacOSX (also 
AdoptOpenJDK)
: * JDK 13-ea+26 was installed on Linux and Windows
: * JDK-13-ea+shipilev-fastdebug (nightly) was updated to yesterday’s 
build) on Linux
: 
:  
: 
: If the SSL errors are really coming from this and Users have to install 
11.0.3 at minimum, we have to mention this in release notes and on the web 
page. Especially we have to tell people to either pay Oracle to get 11.0.3 LTS 
or to use AdoptOpenJDK-Hotspot or Coretto (untested). Of course this is only a 
limitation if you enable TLS, which most people don’t do on their Solr servers.
: 
:  
: 
: Uwe
: 
:  
: 
: -
: 
: Uwe Schindler
: 
: Achterdiek 19, D-28357 Bremen
: 
: https://www.thetaphi.de
: 
: eMail: u...@thetaphi.de
: 
:  
: 
: From: Uwe Schindler  
: Sent: Saturday, June 22, 2019 10:19 AM
: To: Chris Hostetter 
: Cc: Lucene Dev 
: Subject: Re: Need to upgrade jenkins jdk-11 jobs >= 11.0.3 to fix JVM SSL bugs
: 
:  
: 
: Ok, will work on it later today.
: 
: Uwe
: 
: Am June 22, 2019 12:23:17 AM UTC schrieb Chris Hostetter 
mailto:hossman_luc...@fucit.org> >:
: 
: 
: We also need to upgrade the jdk-13 jenkins jobs to at least 13-ea+26, 
: which includes the fix for JDK-8224829...
: 
:   <https://bugs.openjdk.java.net/browse/JDK-8224829> 
https://bugs.openjdk.java.net/browse/JDK-8224829
: 
: : Date: Tue, 18 Jun 2019 14:44:51 -0700 (MST)
: : From: Chris Hostetter < <mailto:hossman_luc...@fucit.org> 
hossman_luc...@fucit.org>
: : To: Uwe Schindler < <mailto:u...@thetaphi.de> u...@thetaphi.de>
: : Cc: Lucene Dev < <mailto:dev@lucene.apache.org> dev@lucene.apache.org>
: : Subject: Need to upgrade jenkins jdk-11 jobs >= 11.0.3 to fix JVM SSL bugs
: : 
: : 
: : TL;DR: Uwe: can you please upgrade the jdk-11 used on the apache lucene 
jenkis
: : jobs and your policeman jenkins jobs to 11.0.3 ?
: : 
: : ---
: : 
: : Dat & I have (coincidently) found ourselves both looking into some (long
: : standing) SSL weirdness that has only ever manifested on java>=11.
: : 
: : Details can be found in SOLR-12988 & SOLR-12990 but the long and short of it
: : is there are at least 2 known OpenJDK bugs in SSL that have been fixed in
: : 11.0.3, which we are seeing evidence of in jenkins builds using 11.0.2
: : 
: :  <https://bugs.openjdk.java.net/browse/JDK-8213202> 
https://bugs.openjdk.java.net/browse/JDK-8213202
: :  <https://bugs.openjdk.java.net/browse/JDK-8212885> 
https://bugs.openjdk.java.net/browse/JDK-8212885 / JDK-8220723
: : 
: : (The nature of these bugs makes it hard -- at least AFAICT -- to try to 
write
: : any "assume" logic to auto-detect if they apply to the current JVM.)
: : 
: : There may in fact still be other SSL related bugs in jdk 11.0.3, but it will
: : be

RE: Need to upgrade jenkins jdk-11 jobs >= 11.0.3 to fix JVM SSL bugs

[Chris Hostetter]

: sorry for the delay, I updated JDK on Policeman Jenkins. But there are 
: some things to mention:

Thanks Uwe, appreciate it

: If the SSL errors are really coming from this and Users have to install 
: 11.0.3 at minimum, we have to mention this in release notes and on the 
: web page. Especially we have to tell people to either pay Oracle to get 

Agreed -- this is being tracked in SOLR-12988, so we should probably keep 
discussion of what/where/how to inform users there -- but for now the 
priority is getting SSL testing re-enabled on jenkins jdk11 (w/11.0.3) 
since it's been silently disabled in the test code for the past 6 months 
... that way we'll have some sanity check that there aren't *OTHER* java11 
+ SSL bugs we haven't found yet, before we tell people "use 11.0.3"


: * JDK 13-ea+26 was installed on Linux and Windows
: * JDK-13-ea+shipilev-fastdebug (nightly) was updated to yesterday’s 
build) on Linux

Unfortunately on friday I discovered that a *new* bug slipped into 
13-ea+26, which garuntees SSL failures in jetty/solr (JDK-8226649) and 
still has no fix, so it looks like we need to skip SSL testing on all 
13-ea builds (even if they are current nightly builds of OpenJDK) for now.

BTW: I still don't really understand what "13-ea+shipilev-fastdebug" is -- 
can you elaborate on that?



-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Need to upgrade jenkins jdk-11 jobs >= 11.0.3 to fix JVM SSL bugs

[Chris Hostetter]

We also need to upgrade the jdk-13 jenkins jobs to at least 13-ea+26, 
which includes the fix for JDK-8224829...

https://bugs.openjdk.java.net/browse/JDK-8224829

: Date: Tue, 18 Jun 2019 14:44:51 -0700 (MST)
: From: Chris Hostetter 
: To: Uwe Schindler 
: Cc: Lucene Dev 
: Subject: Need to upgrade jenkins jdk-11 jobs >= 11.0.3 to fix JVM SSL bugs
: 
: 
: TL;DR: Uwe: can you please upgrade the jdk-11 used on the apache lucene jenkis
: jobs and your policeman jenkins jobs to 11.0.3 ?
: 
: ---
: 
: Dat & I have (coincidently) found ourselves both looking into some (long
: standing) SSL weirdness that has only ever manifested on java>=11.
: 
: Details can be found in SOLR-12988 & SOLR-12990 but the long and short of it
: is there are at least 2 known OpenJDK bugs in SSL that have been fixed in
: 11.0.3, which we are seeing evidence of in jenkins builds using 11.0.2
: 
: https://bugs.openjdk.java.net/browse/JDK-8213202
: https://bugs.openjdk.java.net/browse/JDK-8212885 / JDK-8220723
: 
: (The nature of these bugs makes it hard -- at least AFAICT -- to try to write
: any "assume" logic to auto-detect if they apply to the current JVM.)
: 
: There may in fact still be other SSL related bugs in jdk 11.0.3, but it will
: be hard to know until we at least upgrade to 11.0.3 to see what still fails.
: 
: Uwe / whomever has access: if you could help us out here it would be
: appreciated.
: 
: 
: 
: -Hoss
: http://www.lucidworks.com/
: 

-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-Solr-8.x-Solaris (64bit/jdk1.8.0) - Build # 190 - Still Failing!

[Chris Hostetter]

Gus aparently already figured this out and moved the discussion to the 
realted jira, but to close the loop for folks who may not have seen it...

https://issues.apache.org/jira/browse/SOLR-13534?focusedCommentId=16868656=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16868656


: Date: Thu, 20 Jun 2019 11:39:51 -0400
: From: Gus Heck 
: Reply-To: dev@lucene.apache.org
: To: dev 
: Subject: Re: [JENKINS] Lucene-Solr-8.x-Solaris (64bit/jdk1.8.0) - Build # 190
: - Still Failing!
: 
: hmm I'm seeing this one on 8x locally as well...
: 
:  [ecj-lint] Internal compiler error: java.lang.NullPointerException at
: 
org.eclipse.jdt.internal.compiler.lookup.TypeSystem.getUnannotatedType(TypeSystem.java:167)
:  [ecj-lint] --
:  [ecj-lint] java.lang.NullPointerException
:  [ecj-lint]   at
: 
org.eclipse.jdt.internal.compiler.lookup.TypeSystem.getUnannotatedType(TypeSystem.java:167)
:  [ecj-lint]   at
: 
org.eclipse.jdt.internal.compiler.lookup.TypeSystem$HashedParameterizedTypes.get(TypeSystem.java:97)
:  [ecj-lint]   at
: 
org.eclipse.jdt.internal.compiler.lookup.AnnotatableTypeSystem.getParameterizedType(AnnotatableTypeSystem.java:116)
:  [ecj-lint]   at
: 
org.eclipse.jdt.internal.compiler.lookup.LookupEnvironment.createParameterizedType(LookupEnvironment.java:998)
:  [ecj-lint]   at
: 
org.eclipse.jdt.internal.compiler.lookup.Scope$Substitutor.substitute(Scope.java:508)
:  [ecj-lint]   at
: org.eclipse.jdt.internal.compiler.lookup.Scope.substitute(Scope.java:432)
:  [ecj-lint]   at
: 
org.eclipse.jdt.internal.compiler.lookup.InferenceContext18.reportUncheckedConversions(InferenceContext18.java:1601)
:  [ecj-lint]   at
: 
org.eclipse.jdt.internal.compiler.lookup.InferenceContext18.inferInvocationType(InferenceContext18.java:457)
:  [ecj-lint]   at
: 
org.eclipse.jdt.internal.compiler.lookup.ParameterizedGenericMethodBinding.computeCompatibleMethod18(ParameterizedGenericMethodBinding.java:260)
:  [ecj-lint]   at
: 
org.eclipse.jdt.internal.compiler.lookup.ParameterizedGenericMethodBinding.computeCompatibleMethod(ParameterizedGenericMethodBinding.java:84)
:  [ecj-lint]   at
: 
org.eclipse.jdt.internal.compiler.lookup.Scope.computeCompatibleMethod(Scope.java:743)
:  [ecj-lint]   at
: 
org.eclipse.jdt.internal.compiler.lookup.Scope.computeCompatibleMethod(Scope.java:700)
:  [ecj-lint]   at
: 
org.eclipse.jdt.internal.compiler.lookup.Scope.getStaticFactory(Scope.java:5017)
:  [ecj-lint]   at
: 
org.eclipse.jdt.internal.compiler.ast.AllocationExpression.inferConstructorOfElidedParameterizedType(AllocationExpression.java:571)
:  [ecj-lint]   at
: 
org.eclipse.jdt.internal.compiler.ast.AllocationExpression.resolveType(AllocationExpression.java:460)
:  [ecj-lint]   at
: 
org.eclipse.jdt.internal.compiler.ast.ReturnStatement.resolve(ReturnStatement.java:342)
:  [ecj-lint]   at
: 
org.eclipse.jdt.internal.compiler.ast.LambdaExpression.resolveType(LambdaExpression.java:424)
:  [ecj-lint]   at
: 
org.eclipse.jdt.internal.compiler.ast.LambdaExpression.cachedResolvedCopy(LambdaExpression.java:880)
:  [ecj-lint]   at
: 
org.eclipse.jdt.internal.compiler.ast.LambdaExpression.resolveExpressionExpecting(LambdaExpression.java:909)
:  [ecj-lint]   at
: 
org.eclipse.jdt.internal.compiler.lookup.ConstraintExpressionFormula.reduce(ConstraintExpressionFormula.java:159)
:  [ecj-lint]   at
: 
org.eclipse.jdt.internal.compiler.lookup.BoundSet.reduceOneConstraint(BoundSet.java:949)
:  [ecj-lint]   at
: 
org.eclipse.jdt.internal.compiler.lookup.InferenceContext18.reduce(InferenceContext18.java:885)
:  [ecj-lint]   at
: 
org.eclipse.jdt.internal.compiler.lookup.InferenceContext18.solve(InferenceContext18.java:836)
:  [ecj-lint]   at
: 
org.eclipse.jdt.internal.compiler.lookup.ParameterizedGenericMethodBinding.computeCompatibleMethod18(ParameterizedGenericMethodBinding.java:238)
:  [ecj-lint]   at
: 
org.eclipse.jdt.internal.compiler.lookup.ParameterizedGenericMethodBinding.computeCompatibleMethod(ParameterizedGenericMethodBinding.java:84)
:  [ecj-lint]   at
: 
org.eclipse.jdt.internal.compiler.lookup.Scope.computeCompatibleMethod(Scope.java:743)
:  [ecj-lint]   at
: 
org.eclipse.jdt.internal.compiler.lookup.Scope.computeCompatibleMethod(Scope.java:700)
:  [ecj-lint]   at
: org.eclipse.jdt.internal.compiler.lookup.Scope.findMethod0(Scope.java:1656)
:  [ecj-lint]   at
: org.eclipse.jdt.internal.compiler.lookup.Scope.findMethod(Scope.java:1557)
:  [ecj-lint]   at
: 
org.eclipse.jdt.internal.compiler.lookup.Scope.getImplicitMethod(Scope.java:2524)
:  [ecj-lint]   at
: 
org.eclipse.jdt.internal.compiler.ast.MessageSend.findMethodBinding(MessageSend.java:914)
:  [ecj-lint]   at
: 
org.eclipse.jdt.internal.compiler.ast.MessageSend.resolveType(MessageSend.java:729)
:  [ecj-lint]   at
: 
org.eclipse.jdt.internal.compiler.ast.ReturnStatement.resolve(ReturnStatement.java:342)
:  [ecj-lint]   at
: 
org.eclipse.jdt.internal.compiler.ast.AbstractMethodDeclaration.resolveStatements(AbstractMethodDeclaration.java:634)
:  

Need to upgrade jenkins jdk-11 jobs >= 11.0.3 to fix JVM SSL bugs

[Chris Hostetter]

TL;DR: Uwe: can you please upgrade the jdk-11 used on the apache lucene 
jenkis jobs and your policeman jenkins jobs to 11.0.3 ?


---

Dat & I have (coincidently) found ourselves both looking into some (long 
standing) SSL weirdness that has only ever manifested on java>=11.


Details can be found in SOLR-12988 & SOLR-12990 but the long and short of 
it is there are at least 2 known OpenJDK bugs in SSL that have been 
fixed in 11.0.3, which we are seeing evidence of in jenkins builds using 
11.0.2


https://bugs.openjdk.java.net/browse/JDK-8213202
https://bugs.openjdk.java.net/browse/JDK-8212885 / JDK-8220723

(The nature of these bugs makes it hard -- at least AFAICT -- to try to 
write any "assume" logic to auto-detect if they apply to the current JVM.)


There may in fact still be other SSL related bugs in jdk 11.0.3, but it 
will be hard to know until we at least upgrade to 11.0.3 to see what still 
fails.


Uwe / whomever has access: if you could help us out here it would be 
appreciated.




-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-Solr-Tests-master - Build # 3385 - Still Failing

[Chris Hostetter]

It is sporadic & seemingly unpredictible & has been happening for 
various people on various machines / OSes since the upgrade to java11 on 
master -- see thread "precommit failures" ...

http://mail-archives.apache.org/mod_mbox/lucene-dev/201906.mbox/%3Calpine.DEB.2.11.1906131520060.17915%40tray%3E



: Date: Mon, 17 Jun 2019 15:57:30 -0400
: From: Joel Bernstein 
: Reply-To: dev@lucene.apache.org
: To: lucene dev 
: Subject: Re: [JENKINS] Lucene-Solr-Tests-master - Build # 3385 - Still Failing
: 
: I saw these failures locally when running precommit, After running clean
: and clean-jars the precommit passed. Is there an issue with the jars on
: jenkins?
: 
: 
: 
: Joel Bernstein
: http://joelsolr.blogspot.com/
: 
: 
: On Mon, Jun 17, 2019 at 3:35 PM Apache Jenkins Server <
: jenk...@builds.apache.org> wrote:
: 
: > Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/3385/
: >
: > All tests passed
: >
: > Build Log:
: > [...truncated 65007 lines...]
: > -ecj-javadoc-lint-src:
: > [mkdir] Created dir: /tmp/ecj671470942
: >  [ecj-lint] Compiling 69 source files to /tmp/ecj671470942
: >  [ecj-lint] invalid Class-Path header in manifest of jar file:
: > 
/home/jenkins/.ivy2/cache/org.restlet.jee/org.restlet/jars/org.restlet-2.3.0.jar
: >  [ecj-lint] invalid Class-Path header in manifest of jar file:
: > 
/home/jenkins/.ivy2/cache/org.restlet.jee/org.restlet.ext.servlet/jars/org.restlet.ext.servlet-2.3.0.jar
: >  [ecj-lint] --
: >  [ecj-lint] 1. ERROR in
: > 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/contrib/dataimporthandler/src/java/org/apache/solr/handler/dataimport/JdbcDataSource.java
: > (at line 28)
: >  [ecj-lint] import javax.naming.InitialContext;
: >  [ecj-lint]^^^
: >  [ecj-lint] The type javax.naming.InitialContext is not accessible
: >  [ecj-lint] --
: >  [ecj-lint] 2. ERROR in
: > 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/contrib/dataimporthandler/src/java/org/apache/solr/handler/dataimport/JdbcDataSource.java
: > (at line 29)
: >  [ecj-lint] import javax.naming.NamingException;
: >  [ecj-lint]
: >  [ecj-lint] The type javax.naming.NamingException is not accessible
: >  [ecj-lint] --
: >  [ecj-lint] 3. ERROR in
: > 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/contrib/dataimporthandler/src/java/org/apache/solr/handler/dataimport/JdbcDataSource.java
: > (at line 182)
: >  [ecj-lint] c = getFromJndi(initProps, jndiName);
: >  [ecj-lint] ^^^
: >  [ecj-lint] The method getFromJndi(Properties, String) from the type new
: > Callable(){} refers to the missing type NamingException
: >  [ecj-lint] --
: >  [ecj-lint] 4. ERROR in
: > 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/contrib/dataimporthandler/src/java/org/apache/solr/handler/dataimport/JdbcDataSource.java
: > (at line 245)
: >  [ecj-lint] private Connection getFromJndi(final Properties initProps,
: > final String jndiName) throws NamingException,
: >  [ecj-lint]
: >   ^^^
: >  [ecj-lint] NamingException cannot be resolved to a type
: >  [ecj-lint] --
: >  [ecj-lint] 5. ERROR in
: > 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/contrib/dataimporthandler/src/java/org/apache/solr/handler/dataimport/JdbcDataSource.java
: > (at line 249)
: >  [ecj-lint] InitialContext ctx =  new InitialContext();
: >  [ecj-lint] ^^
: >  [ecj-lint] InitialContext cannot be resolved to a type
: >  [ecj-lint] --
: >  [ecj-lint] 6. ERROR in
: > 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/contrib/dataimporthandler/src/java/org/apache/solr/handler/dataimport/JdbcDataSource.java
: > (at line 249)
: >  [ecj-lint] InitialContext ctx =  new InitialContext();
: >  [ecj-lint]   ^^
: >  [ecj-lint] InitialContext cannot be resolved to a type
: >  [ecj-lint] --
: >  [ecj-lint] 6 problems (6 errors)
: >
: > BUILD FAILED
: > 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/build.xml:634:
: > The following error occurred while executing this line:
: > 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/build.xml:101:
: > The following error occurred while executing this line:
: > 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build.xml:651:
: > The following error occurred while executing this line:
: > 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/common-build.xml:479:
: > The following error occurred while executing this line:
: > 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/common-build.xml:2009:
: > The following error occurred while executing this line:
: > 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/common-build.xml:2048:
: > 

Re: VOTE: Apache Solr Reference Guide for Solr 8.1

[Chris Hostetter]

+1

: Date: Wed, 12 Jun 2019 09:47:39 -0500
: From: Cassandra Targett 
: Reply-To: dev@lucene.apache.org
: To: lucene dev 
: Subject: VOTE: Apache Solr Reference Guide for Solr 8.1
: 
: Please vote to release the Solr Reference Guide for 8.1
: 
: The PDF artifacts can be downloaded from:
: 
https://dist.apache.org/repos/dist/dev/lucene/solr/ref-guide/apache-solr-ref-guide-8.1-RC1/
: 
: $ cat apache-solr-ref-guide-8.1.pdf.sha512
: 
cc76882fb3061fa03d1aa291d9705c1df17f948ff47f3f7d6a18e8ddef907f1c74f078ed482f7f5e04b7c6779a88ad85297cd31ae03570db2acc5930ba2feaf0
:  apache-solr-ref-guide-8.1.pdf
: 
: The HTML version is also available: https://lucene.apache.org/solr/guide/8_1
: 
: Here's my +1.
: 
: Thanks,
: Cassandra
: 

-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: precommit failures

[Chris Hostetter]

So, FWIW: these ecj/precommit ERRORS related to "javax.naming.*" 
packages/classes still seems to be happening periodically 
but unpredictibly & unreliably...

1) speaking persnally: it happens on my machine periodically, even with 
"ant clean precommit" and then goes away the very next time i run "ant 
clean precommit" -- w/o any changes to the source code, working dir, java 
version, etc...  

openjdk version "11.0.2" 2019-01-15
OpenJDK Runtime Environment 18.9 (build 11.0.2+9)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.2+9, mixed mode)

(It's possible it relates to something in the ivy cache, but if so, it can 
aparently "self fix" or "self break" w/o any other java processes running 
on on the machine in the meantime)

2) It also happens periodically to jenkins builds, as recently as 
yesterday, on multiple jenkins clusters and diff build OSs...

https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/5195/
https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/7988/
https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/24208/
https://builds.apache.org/job/Lucene-Solr-Tests-master/3365/
https://builds.apache.org/job/Lucene-Solr-SmokeRelease-master/1360/

...that's just a handful of examples from the past few days, in generally 
ERRORs that start with "The type javax.naming." seem to pop up on at least 
1 jenkins build a day.

The only commonality seems to be builds of *master* using jdk11 
... it doesn't seem to pop up in any 8x builds (even when using 
jdk11  i think because precommit on 8x doesn't do this check anymore?) 
and it doesn't show up in Policeman master builds using jdk12 & jdk13

anybody have any idea WTF is happening here?




: Date: Mon, 06 May 2019 20:25:46 +
: From: Uwe Schindler 
: Reply-To: dev@lucene.apache.org
: To: dev@lucene.apache.org, Erick Erickson 
: Subject: Re: precommit failures
: 
: I am not fully sure if the "java.naming" module is enabled by default in Java 
11. Maybe that's a side effect of some global configuration parameter.
: 
: Is Java version really fully identical including vendor?
: 
: The strange thing is that only ecj breaks. Could it be that you have older 
version of ecj in ant's classpath?
: 
: Uwe
: 
: Am May 6, 2019 7:47:45 PM UTC schrieb Erick Erickson 
:
: >Weirder and weirder. My mac pro precommits successfully, same Java
: >version but my MBP fails every time.
: >
: >> On May 6, 2019, at 9:03 AM, Dawid Weiss 
: >wrote:
: >> 
: >> I had it this morning before committing the fst patch from Mike.
: >> Cleaned the repo, re-ran precommit and it passed... Very strange.
: >> 
: >> D.
: >> 
: >> On Mon, May 6, 2019 at 5:53 PM Erick Erickson
: > wrote:
: >>> 
: >>> 
: >>> Both Kevin Risden and I are seeing:
: >>> 
: >>> [ecj-lint] 1. ERROR in
: 
>/Users/Erick/apache/solrVersions/playspace/solr/contrib/dataimporthandler/src/java/org/apache/solr/handler/dataimport/JdbcDataSource.java
: >(at line 28)
: >>> [ecj-lint] import javax.naming.InitialContext;
: >>> [ecj-lint]^^^
: >>> [ecj-lint] The type javax.naming.InitialContext is not accessible```
: >>> 
: >>> This import hasn’t been changed since 2009.
: >>> 
: >>> I'm using: openjdk version “11.0.2” 2019-01-15
: >>> 
: >>> I tried a fresh clone of master and cleaned the ivy cache, still the
: >same problem. But we can't be the only ones seeing this, any clues?
: >>>
: >-
: >>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
: >>> For additional commands, e-mail: dev-h...@lucene.apache.org
: >>> 
: >> 
: >> -
: >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
: >> For additional commands, e-mail: dev-h...@lucene.apache.org
: >> 
: >
: >
: >-
: >To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
: >For additional commands, e-mail: dev-h...@lucene.apache.org
: 
: --
: Uwe Schindler
: Achterdiek 19, 28357 Bremen
: https://www.thetaphi.de

-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [jira] [Commented] (SOLR-13464) Sporadic Auth + Cloud test failures, probably due to lag in nodes reloading security config

[Chris Hostetter]

: Hoss, I see several of these failures popping up, probably related to 
: timing of the config reload across nodes. Should we as a phase 1 
: introduce a simple sleep to harden those tests and follow up later with 
: APIs that support waiting until config propagates?

Well, I personally refuse to add any sleep calls to any tests -- but 
that's my personal opinion.  You and others may have your own opinions and 
take differnet actions then i would take :)

https://twitter.com/_hossman/status/974743183044128768

: 
: Jan Høydahl
: 
: > 11. mai 2019 kl. 01:46 skrev Hoss Man (JIRA) :
: > 
: > 
: >[ 
https://issues.apache.org/jira/browse/SOLR-13464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16837697#comment-16837697
 ] 
: > 
: > Hoss Man commented on SOLR-13464:
: > -
: > 
: > In theory it would be possible for a test client (or any real production 
client) to poll {{/admin/auth...}} on all/any nodes in a cluster to verify that 
they are using the updated security settings, because the behavior of 
SecurityConfHandlerZk on GET is to read the _cached_ security props from the 
ZkStateReader, so in theory it's only updated once it's been force refreshed by 
the zk watcher ... but this still has 2 problems:
: > # any client doing this would have to be statefull and know what the most 
recent setting(s) change was, so it could assert those specific settings have 
been updated. There's no way for a "dumb" client to simply ask "is your current 
config up to date w/zk". Even if the client directly polled ZK to see what the 
current version is in the authoritative {{/security.json}} for the cluster, the 
"version" info isn't included in the {{GET /admin/auth...}} responses, so it 
would have to do a "deep comparison" of the entire JSON response.
: > # even if client knows what data to expect from a {{GET /admin/auth...}} 
request when polling all/any nodes in the cluster (either from first hand 
knowledge because it was the client that did the last POST, or second hand 
knowledge from querying ZK directly) and even if the expected data is returned 
by every node, that doesn't mean it's in *USE* yet – there is inherient lag 
between when the security conf data is "refreshed" in the ZkStateReader (on 
each node) and when the plugin Object instance are actually initialized and 
become active on each node.
: > 
: > 
: > Here's a strawman proposal for a possible solution to this problem – both 
for use in tests, and for end users, that might want to verify when updated 
settings are in really enabled...
: > # refactor CoreContainer so that methods like {{public AuthorizationPlugin 
getAuthorizationPlugin()}} are deprecated/syntactic sugar for new {{public 
SecurityPluginHolder getAuthorizationPlugin()}} methods so 
that callers can read the znode version used to init the plugin
: > # refactor {{SecurityConfHandler.getPlugin(String)}} to be a 
deprecated/syntactic sugar for a new version that returns 
{{SecurityPluginHolder}}
: > # update {{SecurityConfHandlerZk.getConf}} so that it:
: > ** uses {{getSecurityConfig(true)}} to ensure it reads the most current 
settings from ZK, (instead of the cached copy used by the current code).
: > ** adds the {{SecurityConfig.getVersion()}} number in the response (in 
addition to the config data) ... perhaps as {{key + ".conf.version"}}
: > ** when {{getPlugin(key)}} is non null, include the 
{{SecurityPluginHolder.getVersion()}} in the response ... perhaps as {{key + 
".enabled.version"}}
: > 
: > ...that way a dumb client can easily poll any/all node(s) for 
{{/admin/auth_foo}} until the {{auth_foo.conf.version}} and 
{{auth_foo.enabled.version}} are identical to know when the most recent 
{{auth_foo}} settings in ZK's security.json are actaully in use.
: > 
: > (We could potentially take things even a step further, and add something 
like a {{verify.cluster.version=true|false}} option to SecurityConfHandlerZk, 
that would federate {{GET /admin/auth...}} to every (live?) node in the 
cluster, and include map of nodeName => enabled.version for every node ... 
maybe?)
: > 
: > Thoughts?
: > 
: >> Sporadic Auth + Cloud test failures, probably due to lag in nodes 
reloading security config
: >> 
---
: >> 
: >>Key: SOLR-13464
: >>URL: https://issues.apache.org/jira/browse/SOLR-13464
: >>Project: Solr
: >> Issue Type: Bug
: >> Security Level: Public(Default Security Level. Issues are Public) 
: >>   Reporter: Hoss Man
: >>   Priority: Major
: >> 
: >> I've been investigating some sporadic and hard to reproduce test failures 
related to authentication in cloud mode, and i *think* (but have not directly 
verified) that the common cause is that after uses one of the 
{{/admin/auth...}} handlers to update some setting, there is an inherient and 
unpredictible delay (due 

Re: VOTE: Apache Solr Reference Guide for Solr 8.0

[Chris Hostetter]

+1

: Date: Fri, 31 May 2019 09:24:23 -0500
: From: Cassandra Targett 
: Reply-To: dev@lucene.apache.org
: To: lucene dev 
: Subject: VOTE: Apache Solr Reference Guide for Solr 8.0
: 
: Please vote to release the Solr Reference Guide for 8.0.
: 
: The PDF artifacts can be downloaded from:
: 
https://dist.apache.org/repos/dist/dev/lucene/solr/ref-guide/apache-solr-ref-guide-8.0-RC1/
: 
: $ cat apache-solr-ref-guide-8.0.pdf.sha512
: 
c3d01089cb68fdbaa49dc94483a2643fba71ff9900840fbb1a6cc731f2116a9ce9994ab8b832429948f61c6caa7a52f1abfbc8c479c6958644076a8bf99a62f9
:  apache-solr-ref-guide-8.0.pdf
: 
: The online version is available at:
: https://lucene.apache.org/solr/guide/8_0/
: 
: Here's my +1.
: 
: Cassandra
: 
: PS - The 8.1 Guide will follow along hopefully by next week.
: 

-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: ReleaseWizard tool

[Chris Hostetter]

: I tested relasedocmaker and it digs up all the JIRA issues marked as RESOLVED 
for the version and creates two files.
: CHANGELOG.md simply lists all issues under headings IMPROVEMENTS, BUG FIXES 
etc
: One problem I found with how the CHANGELOG works is that it adds all issues 
having the version in fixVersion, even if the feature
: was already released in an earlier version. That is because of the way we use 
JIRA fixVersion, adding both e.g. "master (9.0)" and "8.2"
: at the same time, even if we know that 8.2 is the version the feature will be 
released. If we stop always adding "master" to fixVersion
: but strive to keep it a list of version the feature/bugfix is FIRST 
introduced, then this tool will do the correct job.

The reason we (should always) track every fixVersion based on where things 
are commited & backported is because of how important it is from the stand 
point of knowing when exactly something was fixed, compared to where it 
might have *needed* to be fixed.  

A bug might be found just before the 8.1.0 release that affect 7.7.0, 
8.0.0, and the current branch_8x but not master because the underlying 
functionality was deprecated in 8x & removed from master -- the fix would 
be committed to branch_8x and backported to branch_8_0 and branch_7_7 for 
inclusion in 7.7.1 -- but maybe no one has bothered with w/an 8.0.1, and 
meanwhile branch_8_1 is forked from branch_8x and already has the fix.

if we *only* record fixVersion=7.7.1, then that can misslead people who 
don't know the timing/history of the jira into thinking that if it was 
fixed in 7.7.y then maybe/probably/who-knows-if it is issue in 8.x.y.  
Having fixVersion=7.7.1,8.0.1,8.1.0 helps communicate several things...
 
* if you are using 7.y.x lower then 7.7.1 you might be affected by this
* if you are using 8.0.y lower then 8.0.1 you might be affected by this
  - since you can see in jira that 8.0.1 is unreleased:
- if/when 8.0.1 is released, it will include this fix
* if you are using 8.x.y lower then 8.1.0 then you might be affected by this

If we *only* use fixVersion="lowest version released" it wouldn't help 
people answer the question "Does this bug exist in 8.w.v if all i know is that 
the _first_ version it was fixed in was 7.x.y?"

...not to mention the complexity involved in _older_ bug fixe releases 
happening *after* more recent feature releases with the same fix (ie: 
7.7.1 being released after 8.1.0.

(a more rigorous use of the "affectsVersion", eg: 
affectsVersion="7.7.0,7.7.1,...,7.7.0,8.0.0", can help mitigate *some* 
of this confusion -- by implying that the issue does *not* affect 8.1.0 
since it isn't listed there -- but that's a lot more tedious to maintain 
then just ensuring that you add a fixVersion for every place you commit)


-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

RE: Change of master to Java 11 tomorrow, 2019-04-16, 12:00 UTC

[Chris Hostetter]

: > - The HTML linter based on HTML Tidy was disabled until we found a
: > replacement (no HTML 5 support - Java 11 produces HTML5 Javadocs). So it
: > might happen that bad javadocs are not detected in master but after merge
: > to 8.x branch they might suddenly appear.
: 
: Maybe we can use the infrastriucture using ASCIIdoctor of the Solr Ref guide 
here? It seems to also do link / HTML checks.

The code in the ref-guide's CheckLinksAndAnchors.java is *very* 
specialized around dealing with the specific problems that exist in 
asciidoctor when building merged/monolithic docs (like our PDF) from 
multiple source documents -- ie: only a small bit of it is about checking 
links, most of it is about checking for duplicate anchors; and even the 
link checking makes a lot of assumptions about the structure/conventions 
of generated asciidoctor output.

If someone wanted to build a quick and dirty javadoc link checker, the 
jsoup parsing code in CheckLinksAndAnchors.java might be a good starting 
place, but it would probably be eaiser to build a new one then trying to 
generalize what's there to work on arbitrary html.




-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

RE: Change of master to Java 11 tomorrow, 2019-04-16, 12:00 UTC

[Chris Hostetter]

: The master branch was updated to use Java 11 - all Jenkins Jobs are back 
online! Let's see if additional Jenkins failures occur.

FYI: first oddity i noticed is coming from some groovy code that seems 
to run as part of "common.test" ?

$ git clean -fx && cd solr/core/ && ant test -Dtestcase=TestCloudConsistency
...
   [junit4] Tests summary: 1 suite, 2 tests
 [echo] 5 slowest tests:
[junit4:tophints] 315.23s | org.apache.solr.cloud.ShardSplitTest
[junit4:tophints] 260.44s | org.apache.solr.search.facet.TestJsonFacets
[junit4:tophints] 208.15s | org.apache.solr.cloud.TestHossSanity
[junit4:tophints] 186.80s | org.apache.solr.cloud.api.collections.ShardSplitTest
[junit4:tophints] 125.64s | org.apache.solr.cloud.OnlyLeaderIndexesTest

-check-totals:
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by 
org.codehaus.groovy.reflection.CachedClass 
(file:/home/hossman/.ivy2/cache/org.codehaus.groovy/groovy-all/jars/groovy-all-2.4.15.jar)
 to method java.lang.Object.finalize()
WARNING: Please consider reporting this to the maintainers of 
org.codehaus.groovy.reflection.CachedClass
WARNING: Use --illegal-access=warn to enable warnings of further illegal 
reflective access operations
WARNING: All illegal access operations will be denied in a future release

common.test:

BUILD SUCCESSFUL
Total time: 1 minute 31 seconds

I'm using...

hossman@tray:~/lucene/dev/solr/core [master] $ java -version
openjdk version "11.0.2" 2019-01-15
OpenJDK Runtime Environment 18.9 (build 11.0.2+9)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.2+9, mixed mode)

...which i believe is the latest?

-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-13294) TestSQLHandler failures on windows jenkins machines

[Chris Hostetter (Unused) (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-13294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16796618#comment-16796618
 ] 

Chris Hostetter (Unused) commented on SOLR-13294:
-

I’m not near a computer, replying on my phone - feel free to paste into jira...

I suspect there are 2 bugs: the test flaw regarding strict order assumption 
that maybe is just a fluke only on windows - not sure the exact cause ther -  
and a test setup/tear down bug that causes the same directory to be used when 
tests.dups kicks in and the same jam runs both, but the field rAndomization 
picks a diff field type? that’s maybe exacerbated on windows by not being able 
to delete the directory while still open. 

-Hoss



> TestSQLHandler failures on windows jenkins machines
> ---
>
> Key: SOLR-13294
> URL: https://issues.apache.org/jira/browse/SOLR-13294
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Hoss Man
>Assignee: Joel Bernstein
>Priority: Major
>
> _Windows_ jenkins builds frequently - but _not always_ - fail on 
> {{TestSQLHandler}} @ L236
> In cases where a windows jenkins build finds a failing seed for 
> {{TestSQLHandler}}, and the same jenkins build attempts to reproduce using 
> that seed, it reliably encounters a *different* failure earlier in the test 
> (related to docValues being missing from a sort field).
> These seeds do not fail for me when attempted on a Linux machine, and my own 
> attempts @ beasting on linux hasn't turned up any similar failures.
> Here's an example from jenkins - the exact same pattern has occured in other 
> windows jenkins builds on other branches at the exact same asserts..
> [https://jenkins.thetaphi.de/view/Lucene-Solr/job/Lucene-Solr-8.0-Windows/57/]
> {noformat}
> Using Java: 32bit/jdk1.8.0_172 -server -XX:+UseConcMarkSweepGC
> ...
> Checking out Revision 0376bc0052a53480ecb2edea7dfe58298bda5deb 
> (refs/remotes/origin/branch_8_0)
> ...
>[junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestSQLHandler 
> -Dtests.method=doTest -Dtests.seed=EEE2628F22F5C82A -Dtests.slow=true 
> -Dtests.locale=id -Dtests.timezone=BST -Dtests.asserts=true 
> -Dtests.file.encoding=ISO-8859-1
>[junit4] FAILURE 23.3s J0 | TestSQLHandler.doTest <<<
>[junit4]> Throwable #1: java.lang.AssertionError
>[junit4]>at 
> __randomizedtesting.SeedInfo.seed([EEE2628F22F5C82A:49A6DA2B4F4EDB93]:0)
>[junit4]>at 
> org.apache.solr.handler.TestSQLHandler.testBasicSelect(TestSQLHandler.java:236)
>[junit4]>at 
> org.apache.solr.handler.TestSQLHandler.doTest(TestSQLHandler.java:93)
>[junit4]>at 
> org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1082)
>[junit4]>at 
> org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:1054)
>[junit4]>at java.lang.Thread.run(Thread.java:748)
> ...
>[junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestSQLHandler 
> -Dtests.method=doTest -Dtests.seed=EEE2628F22F5C82A -Dtests.slow=true 
> -Dtests.badapples=true -Dtests.locale=id -Dtests.timezone=BST 
> -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
>[junit4] ERROR   20.8s J0 | TestSQLHandler.doTest <<<
>[junit4]> Throwable #1: java.io.IOException: --> 
> http://127.0.0.1:61309/collection1_shard2_replica_n1:Failed to execute 
> sqlQuery 'select id, field_i, str_s, field_i_p, field_f_p, field_d_p, 
> field_l_p from collection1 where (text='()' OR text='') AND 
> text='' order by field_i desc' against JDBC connection 
> 'jdbc:calcitesolr:'.
>[junit4]> Error while executing SQL "select id, field_i, str_s, 
> field_i_p, field_f_p, field_d_p, field_l_p from collection1 where 
> (text='()' OR text='') AND text='' order by field_i desc": 
> java.io.IOException: java.util.concurrent.ExecutionException: 
> java.io.IOException: --> 
> http://127.0.0.1:61309/collection1_shard2_replica_n1/:id{type=string,properties=indexed,stored,sortMissingLast,uninvertible}
>  must have DocValues to use this feature.
>[junit4]>at 
> __randomizedtesting.SeedInfo.seed([EEE2628F22F5C82A:49A6DA2B4F4EDB93]:0)
>[junit4]>at 
> org.apache.solr.client.solrj.io.stream.SolrStream.read(SolrStream.java:215)
>[junit4]>at 
> org.apache.s

FYI: gitbox web URLs are deliberately 403ing ... was: [jira] [Updated] (INFRA-17956) gitbox web ui returning 403 for some types of requests (fwd)

[Chris Hostetter]

FYI: The gitbox web UI is deliberately returning 403 most URLs (besides 
the basic "commit summary") as a spam/abuse prevention mechanism


The official recommendation from infra is "Browse / review commits on 
github"


-Hoss

-- Forwarded message --
Date: Mon, 11 Mar 2019 01:10:00 + (UTC)
From: "Chris Lambertus (JIRA)" 
To: hossman_apachej...@fucit.org
Subject: [jira] [Updated] (INFRA-17956) gitbox web ui returning 403 for some
types of requests


 [ 
https://issues.apache.org/jira/browse/INFRA-17956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chris Lambertus updated INFRA-17956:

Status: Waiting for user  (was: Waiting for Infra)

Yes, we had to disable a number of features due to extreme abuse that was 
taking gitbox down. Please use github's UI for now.



gitbox web ui returning 403 for some types of requests
--

Key: INFRA-17956
URL: https://issues.apache.org/jira/browse/INFRA-17956
Project: Infrastructure
 Issue Type: Bug
 Components: GitBox
   Reporter: Hoss Man
   Priority: Major

using the gitbox web ui to browse commit info returns 403 error codes for some 
types of requests (ie: things that the gitbox web ui provides links to when 
viewing a particular commit)
### examples of HTTP requests that work
# view a commit
https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=b8d569a
# view a file as of a commit
https://gitbox.apache.org/repos/asf?p=lucene-solr.git;a=blob;f=solr/core/src/java/org/apache/solr/client/solrj/embedded/JettySolrRunner.java;h=ba94104bba12ff6e424fdd43b766167d8ae12e8b;hb=b8d569a
# view entire diff for a commit
https://gitbox.apache.org/repos/asf?p=lucene-solr.git;a=commitdiff;h=b8d569a
 examples of HTTP requests that return 403 forbidden
# view commit as patch
https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=b8d569a
# view diff of indvidual file mentioned in commit
https://gitbox.apache.org/repos/asf?p=lucene-solr.git;a=blobdiff;f=solr/CHANGES.txt;h=0d3b41e955a4e9d0e58c9f4fb6a20422a65631e4;hp=bd04156ba16b51ad243f07fd12344dd018202b7c;hb=b8d569a;hpb=a940c40b181e00478b63bb2b77e87cf6aabfc725
# view history of file as of a specific commit
https://gitbox.apache.org/repos/asf?p=lucene-solr.git;a=history;f=solr/core/src/java/org/apache/solr/client/solrj/embedded/JettySolrRunner.java;hb=b8d569a




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-Solr-NightlyTests-8.0 - Build # 7 - Unstable

[Chris Hostetter]

Seems like a test (or code) bug introduced when this test was 
added by LUCENE-8585 ...

https://issues.apache.org/jira/browse/LUCENE-8585



: Date: Tue, 26 Feb 2019 08:59:25 + (UTC)
: From: Apache Jenkins Server 
: Reply-To: dev@lucene.apache.org
: To: dev@lucene.apache.org
: Subject: [JENKINS] Lucene-Solr-NightlyTests-8.0 - Build # 7 - Unstable
: 
: Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-8.0/7/
: 
: 8 tests failed.
: FAILED:  
org.apache.lucene.codecs.lucene70.TestLucene70NormsFormat.testFewValues
: 
: Error Message:
: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.0/checkout/lucene/build/backward-codecs/test/J1/temp/lucene.codecs.lucene70.TestLucene70NormsFormat_C3613DC62817C401-001/index-NIOFSDirectory-001/_1k.fdx:
 Too many open files
: 
: Stack Trace:
: java.nio.file.FileSystemException: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.0/checkout/lucene/build/backward-codecs/test/J1/temp/lucene.codecs.lucene70.TestLucene70NormsFormat_C3613DC62817C401-001/index-NIOFSDirectory-001/_1k.fdx:
 Too many open files
:   at 
__randomizedtesting.SeedInfo.seed([C3613DC62817C401:E1CC12DA3AA6A4FA]:0)
:   at 
org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:48)
:   at 
org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:81)
:   at 
org.apache.lucene.mockfile.HandleTrackingFS.newFileChannel(HandleTrackingFS.java:197)
:   at 
org.apache.lucene.mockfile.FilterFileSystemProvider.newFileChannel(FilterFileSystemProvider.java:202)
:   at java.nio.channels.FileChannel.open(FileChannel.java:287)
:   at java.nio.channels.FileChannel.open(FileChannel.java:335)
:   at 
org.apache.lucene.store.NIOFSDirectory.openInput(NIOFSDirectory.java:81)
:   at 
org.apache.lucene.util.LuceneTestCase.slowFileExists(LuceneTestCase.java:2801)
:   at 
org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:747)
:   at 
org.apache.lucene.store.Directory.openChecksumInput(Directory.java:157)
:   at 
org.apache.lucene.store.MockDirectoryWrapper.openChecksumInput(MockDirectoryWrapper.java:1069)
:   at 
org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.(CompressingStoredFieldsReader.java:128)
:   at 
org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsReader(CompressingStoredFieldsFormat.java:121)
:   at 
org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsReader(Lucene50StoredFieldsFormat.java:173)
:   at 
org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:126)
:   at org.apache.lucene.index.SegmentReader.(SegmentReader.java:83)
:   at 
org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:66)
:   at 
org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:58)
:   at 
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:680)
:   at 
org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:81)
:   at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:63)
:   at 
org.apache.lucene.index.BaseNormsFormatTestCase.doTestNormsVersusDocValues(BaseNormsFormatTestCase.java:494)
:   at 
org.apache.lucene.index.BaseNormsFormatTestCase.testFewValues(BaseNormsFormatTestCase.java:181)
:   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
:   at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
:   at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
:   at java.lang.reflect.Method.invoke(Method.java:498)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
:   at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
:   at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
:   at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
:   at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
:   at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
:   at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
:   at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
:   at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
:   at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
:  

Re: [JENKINS-EA] Lucene-Solr-8.0-Linux (64bit/jdk-13-ea+8) - Build # 196 - Unstable!

[Chris Hostetter]

: Do we need to disable the JDK 13 builds for 8.0, or change the assumptions 
here?  Does Mockito work with JDK 13?

The message about Mockito is a red-herring -- that comes from an failed 
*assumption* which is already checking if the current version of mockito 
works or not on the current JVM in use.  In this case it doesn't, so the 
test is trying to IGNORE itself ... the problem is that the test is 
written poorly, and after the assumeWorkingMockito() call in a 
@BeforeClass throws a perfectly fine AssumptionViolatedException, the 
@AfterClass method is called, and it naivly expects some variables will 
have always been initialized -- and in this case they haven't been because 
of the previously failed assumption

so the "real" failure is simply the NullPointerException -- the 
AssumptionViolatedException is just an "fyi, this is a suite level thing 
that happened prior to the final failure"

I'm testing & commiting & backporting a fix to the test now...


: 
: > On 19 Feb 2019, at 16:19, Policeman Jenkins Server  
wrote:
: > 
: > Build: https://jenkins.thetaphi.de/job/Lucene-Solr-8.0-Linux/196/
: > Java: 64bit/jdk-13-ea+8 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC
: > 
: > 8 tests failed.
: > FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.OverseerTest
: > 
: > Error Message:
: > SOLR-11606: ByteBuddy used by Mockito is not working with this JVM version.
: > 
: > Stack Trace:
: > org.junit.AssumptionViolatedException: SOLR-11606: ByteBuddy used by 
Mockito is not working with this JVM version.
: > at __randomizedtesting.SeedInfo.seed([F6C2057E27AC7E66]:0)
: > at 
com.carrotsearch.randomizedtesting.RandomizedTest.assumeNoException(RandomizedTest.java:742)
: > at 
org.apache.solr.SolrTestCaseJ4.assumeWorkingMockito(SolrTestCaseJ4.java:365)
: > at org.apache.solr.cloud.OverseerTest.beforeClass(OverseerTest.java:284)
: > at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
: > at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
: > at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
: > at java.base/java.lang.reflect.Method.invoke(Method.java:567)
: > at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
: > at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:878)
: > at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
: > at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: > at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
: > at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
: > at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: > at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
: > at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
: > at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
: > at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: > at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: > at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
: > at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
: > at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
: > at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
: > at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: > at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
: > at java.base/java.lang.Thread.run(Thread.java:835)
: > Caused by: java.lang.IllegalArgumentException: Unknown Java version: 13
: > at 
net.bytebuddy.ClassFileVersion.ofJavaVersion(ClassFileVersion.java:210)
: > at 
net.bytebuddy.ClassFileVersion$VersionLocator$ForJava9CapableVm.locate(ClassFileVersion.java:462)
: > at net.bytebuddy.ClassFileVersion.ofThisVm(ClassFileVersion.java:223)
: > at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
: > at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
: > at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
: > at 

Re: Lucene/Solr 8.0

[Chris Hostetter]

: OK, Christmas caught up with me a bit… I’ve just created a branch for 8x 
: from master, and am in the process of updating the master branch to 
: version 9.  New commits that should be included in the 8.0 release 
: should also be back-ported to branch_8x from master.

It looks like someone renamed the "master (8.0)" version of SOLR & LUCENE 
in Jira to "master (9.0)" but IIUC that's definitely *NOT* correct ... 
because it means all the stuff that's been committed to origin/master over 
the past X months won't be listed as "fixed in '8.0'" when people look 
at jira in the future.

I'm pretty sure "master (8.0)" should have been renamed "8.0" and a completely 
new version (with a new internal ID in jira) should have been added for 
"master (9.0)"

Right?

(In the meantime, it seems folks have already added new "8.0" 
versions for SOLR/LUCENE to Jira, which have a handful of issues mapped to 
them, that will need cleaned up)



: > >> >>
: > >> >> On Wed, Oct 17, 2018 at 12:13 PM jim ferenczi 
mailto:jim.feren...@gmail.com>> wrote:
: > >> >>>
: > >> >>> Ok thanks for answering.
: > >> >>>
: > >> >>> > - I think Solr needs a couple more weeks since the work Dat is 
doing isn't quite done yet.
: > >> >>>
: > >> >>> We can wait a few more weeks to create the branch but I don't 
think that one action (creating the branch) prevents the other (the work Dat is 
doing).
: > >> >>> HTTP/2 is one of the blocker for the release but it can be done 
in master and backported to the appropriate branch as any other feature ? We 
just need an issue with the blocker label to ensure that
: > >> >>> we don't miss it ;). Creating the branch early would also help 
in case you don't want to release all the work at once in 8.0.0.
: > >> >>> Next week was just a proposal, what I meant was soon because we 
target a release in a few months.
: > >> >>>
: > >> >>>
: > >> >>> Le mer. 17 oct. 2018 à 17:52, Cassandra Targett 
mailto:casstarg...@gmail.com>> a écrit :
: > >> 
: > >>  IMO next week is a bit too soon for the branch - I think Solr 
needs a couple more weeks since the work Dat is doing isn't quite done yet.
: > >> 
: > >>  Solr needs the HTTP/2 work Dat has been doing, and he told me 
yesterday he feels it is nearly ready to be merged into master. However, it 
does require a new release of Jetty to Solr is able to retain Kerberos 
authentication support (Dat has been working with that team to help test the 
changes Jetty needs to support Kerberos with HTTP/2). They should get that 
release out soon, but we are dependent on them a little bit.
: > >> 
: > >>  He can hopefully reply with more details on his status and what 
else needs to be done.
: > >> 
: > >>  Once Dat merges his work, IMO we should leave it in master for 
a little bit. While he has been beasting and testing with Jenkins as he goes 
along, I think it would be good to have all the regular master builds work on 
it for a little bit also.
: > >> 
: > >>  Of the other blockers, the only other large-ish one is to fully 
remove Trie* fields, which some of us also discussed yesterday and it seemed we 
concluded that Solr isn't really ready to do that. The performance issues with 
single value lookups are a major obstacle. It would be nice if someone with a 
bit more experience with that could comment in the issue (SOLR-12632) and/or 
unmark it as a blocker.
: > >> 
: > >>  Cassandra
: > >> 
: > >>  On Wed, Oct 17, 2018 at 8:38 AM Erick Erickson 
mailto:erickerick...@gmail.com>> wrote:
: > >> >
: > >> > I find 9 open blockers for 8.0:
: > >> >
: > >> > 
https://issues.apache.org/jira/issues/?jql=project%20%3D%20SOLR%20AND%20priority%20%3D%20Blocker%20AND%20status%20%3D%20OPEN
 

: > >> >
: > >> > As David mentioned, many of the SOlr committers are at 
Activate, which
: > >> > ends Thursday so feedback (and work) may be a bit delayed.
: > >> > On Wed, Oct 17, 2018 at 8:11 AM David Smiley 
mailto:david.w.smi...@gmail.com>> wrote:
: > >> > >
: > >> > > Hi,
: > >> > >
: > >> > > Thanks for volunteering to do the 8.0 release Jim!
: > >> > >
: > >> > > Many of us are at the Activate Conference in Montreal.  We 
had a committers meeting where we discussed some of the blockers.  I think only 
a couple items were raised.  I'll leave Dat to discuss the one on HTTP2.  On 
the Solr nested docs front, I articulated one and we mostly came to a decision 
on how to do it.  It's not "hard" just a matter of how to hook in some 
functionality so that it's user-friendly.  I'll file an issue for this.  
Inexplicably I'm sheepish about marking issues "blocker" but I shouldn't be.  

Re: VOTE: Solr Reference Guide for Solr 7.6 RC1

[Chris Hostetter]

+1

: Date: Mon, 10 Dec 2018 09:36:46 -0600
: From: Cassandra Targett 
: Reply-To: dev@lucene.apache.org
: To: dev@lucene.apache.org
: Subject: VOTE: Solr Reference Guide for Solr 7.6 RC1
: 
: Please vote for the release of the Solr Reference Guide for 7.6:
: 
: The PDF artifacts can be downloaded from:
: 
https://dist.apache.org/repos/dist/dev/lucene/solr/ref-guide/apache-solr-ref-guide-7.6-RC1/
: 
: $ cat apache-solr-ref-guide-7.6.pdf.sha512
: 
6fb7e3bc61f1dfae9d9d8a7af64e15dd68f8bbe903157a96bf3842ee476dc6238776a3245a93808fcc8d9c015b982f4afc42c9e2c67e74c28f1bca2974778c03
: apache-solr-ref-guide-7.6.pdf
: 
: For this release, the PDF is up to 1415 pages (16.2Mb).
: 
: The HTML version is also available at
: https://lucene.apache.org/solr/guide/7_6.
: 
: Here's my +1.
: 
: Thanks,
: Cassandra
: 

-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Uptick in nightly failures of TestLatLonLine*Queries.testRandomBig

[Chris Hostetter]

I don't know what might be the cause/corrolation of this, and i haven't 
tried to investigate the logs in depth, but i thought i should point it out...


In the past week, the failure rate of these 2 nightly tests has increased 
~20-25% as compared to the prior 3 (calendar) weeks...


Class: org.apache.lucene.document.TestLatLonLineShapeQueries
Method: testRandomBig
Historic Failures: 22.35% (19 / 85)
Recent Failures: 46.67% (7 / 15)

Class: org.apache.lucene.document.TestLatLonPolygonShapeQueries
Method: testRandomBig
Historic Failures: 14.10% (11 / 78)
Recent Failures: 35.71% (5 / 14)

...all of the recent failures have come from sarowe's jenkins box ... but 
i'm not sure if that indicates a problem with running these tests on his 
machine, or if he's just running nightly tests more often then other 
jenkins hosts?


http://fucit.org/solr-jenkins-reports/suspicious-failure-report.html



-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org