Re: [apache/lucene] Run failed: Run nightly: buildAndPushRelease and smokeTestRelease.py - main (df154cd)

[Dawid Weiss]

+1. I think the idea of those smoketester checks was to only have it for
actually released versions.

On Fri, Apr 5, 2024 at 1:30 PM Benjamin Trent  wrote:

> Hmm, yeah. Honestly, I am not sure what to do about this either. I am
> going to remove the 9.10.1 versioning from all branches but 9.10 (it's
> there to capture the next bugfix).
>
> I thought I was doing something helpful, but I guess I was a little too
> eager.
>
> On Fri, Apr 5, 2024 at 3:13 AM Dawid Weiss  wrote:
>
>>
>> Hi Ben,
>>
>> This fails in the smoke tester - failed last night too, so it reproduces.
>>
>> https://github.com/apache/lucene/actions/workflows/run-nightly-smoketester.yml
>>
>> I looked it up to getAllLuceneReleases in the smoke tester script, which
>> in turn lists all releases available at:
>> https://archive.apache.org/dist/lucene/java/
>>
>> version 9.10.1 isn't there so it complains with:
>> RuntimeError: tested version=9.10.1 but it was not released?
>>
>> I'm not sure how to deal with yet-unreleased minor versions myself!
>>
>> D.
>>
>> On Thu, Apr 4, 2024 at 4:20 PM Benjamin Trent 
>> wrote:
>>
>>> This seems related to us forgetting to make the back-compat indices &
>>> versions when 9.10.1 was released and me adding them later.
>>>
>>> I have since added the 9.10.1 to Version.java and version.txt in main
>>> and 9x. Now, both main and 9x have the back-compat indices (these changes
>>> were not at the same time, and cause a separate build failure noticed by
>>> Mike M.).
>>>
>>> But this failure commit on
>>> main df154cdc2288a33747edb8849509ea5c3cbf792e, contains both of my changes
>>> (the 9.10.1 version & back compat indices).
>>>
>>> Was there something else forgotten during this bugfix release that we
>>> need to address? I am very new to the back-compat and release logic in
>>> Lucene and I am eager to learn.
>>>
>>> On Thu, Apr 4, 2024 at 9:43 AM Dawid Weiss 
>>> wrote:
>>>

 https://github.com/apache/lucene/actions/runs/8548297347/job/23421799032

 This smoketester run failed with:

 > RuntimeError: tested version=9.10.1 but it was not released?

 I guess it's not a hiccup but something recent?

 On Thu, Apr 4, 2024 at 3:24 AM Dawid Weiss 
 wrote:

>
> [image: GitHub] [apache/lucene] Run nightly: buildAndPushRelease and
> smokeTestRelease.py workflow run
>
>   Run nightly: buildAndPushRelease and smokeTestRelease.py: Some jobs
> were not successful
>
> View workflow run
> 
>
> [image: Smoke test release on jdk 21, ubuntu-latest]
>
> *Run nightly: buildAndPushRelease and smokeTestRelease.py* / Smoke
> test release on jdk 21, ubuntu-latest
> Failed in 9 minutes and 22 seconds
> [image: annotations for Run nightly: buildAndPushRelease and
> smokeTestRelease.py / Smoke test release on jdk 21, ubuntu-latest] 1
> 
> [image: Smoke test release on jdk 22-ea, ubuntu-latest]
>
> *Run nightly: buildAndPushRelease and smokeTestRelease.py* / Smoke
> test release on jdk 22-ea, ubuntu-latest
> Cancelled
> [image: annotations for Run nightly: buildAndPushRelease and
> smokeTestRelease.py / Smoke test release on jdk 22-ea, ubuntu-latest] 2
> 
>
>
>
> —
> You are receiving this because you are subscribed to this thread.
> Manage your GitHub Actions notifications
> 
>
>
> GitHub, Inc. ・88 Colin P Kelly Jr Street ・San Francisco, CA 94107
>
>

Re: [apache/lucene] Run failed: Run nightly: buildAndPushRelease and smokeTestRelease.py - main (df154cd)

[Benjamin Trent]

Hmm, yeah. Honestly, I am not sure what to do about this either. I am going
to remove the 9.10.1 versioning from all branches but 9.10 (it's there to
capture the next bugfix).

I thought I was doing something helpful, but I guess I was a little too
eager.

On Fri, Apr 5, 2024 at 3:13 AM Dawid Weiss  wrote:

>
> Hi Ben,
>
> This fails in the smoke tester - failed last night too, so it reproduces.
>
> https://github.com/apache/lucene/actions/workflows/run-nightly-smoketester.yml
>
> I looked it up to getAllLuceneReleases in the smoke tester script, which
> in turn lists all releases available at:
> https://archive.apache.org/dist/lucene/java/
>
> version 9.10.1 isn't there so it complains with:
> RuntimeError: tested version=9.10.1 but it was not released?
>
> I'm not sure how to deal with yet-unreleased minor versions myself!
>
> D.
>
> On Thu, Apr 4, 2024 at 4:20 PM Benjamin Trent 
> wrote:
>
>> This seems related to us forgetting to make the back-compat indices &
>> versions when 9.10.1 was released and me adding them later.
>>
>> I have since added the 9.10.1 to Version.java and version.txt in main and
>> 9x. Now, both main and 9x have the back-compat indices (these changes were
>> not at the same time, and cause a separate build failure noticed by Mike
>> M.).
>>
>> But this failure commit on main df154cdc2288a33747edb8849509ea5c3cbf792e,
>> contains both of my changes (the 9.10.1 version & back compat indices).
>>
>> Was there something else forgotten during this bugfix release that we
>> need to address? I am very new to the back-compat and release logic in
>> Lucene and I am eager to learn.
>>
>> On Thu, Apr 4, 2024 at 9:43 AM Dawid Weiss  wrote:
>>
>>>
>>> https://github.com/apache/lucene/actions/runs/8548297347/job/23421799032
>>>
>>> This smoketester run failed with:
>>>
>>> > RuntimeError: tested version=9.10.1 but it was not released?
>>>
>>> I guess it's not a hiccup but something recent?
>>>
>>> On Thu, Apr 4, 2024 at 3:24 AM Dawid Weiss 
>>> wrote:
>>>

 [image: GitHub] [apache/lucene] Run nightly: buildAndPushRelease and
 smokeTestRelease.py workflow run

   Run nightly: buildAndPushRelease and smokeTestRelease.py: Some jobs
 were not successful

 View workflow run
 

 [image: Smoke test release on jdk 21, ubuntu-latest]

 *Run nightly: buildAndPushRelease and smokeTestRelease.py* / Smoke
 test release on jdk 21, ubuntu-latest
 Failed in 9 minutes and 22 seconds
 [image: annotations for Run nightly: buildAndPushRelease and
 smokeTestRelease.py / Smoke test release on jdk 21, ubuntu-latest] 1
 
 [image: Smoke test release on jdk 22-ea, ubuntu-latest]

 *Run nightly: buildAndPushRelease and smokeTestRelease.py* / Smoke
 test release on jdk 22-ea, ubuntu-latest
 Cancelled
 [image: annotations for Run nightly: buildAndPushRelease and
 smokeTestRelease.py / Smoke test release on jdk 22-ea, ubuntu-latest] 2
 



 —
 You are receiving this because you are subscribed to this thread.
 Manage your GitHub Actions notifications
 


 GitHub, Inc. ・88 Colin P Kelly Jr Street ・San Francisco, CA 94107


>>>

Re: [apache/lucene] Run failed: Run nightly: buildAndPushRelease and smokeTestRelease.py - main (df154cd)

[Dawid Weiss]

Hi Ben,

This fails in the smoke tester - failed last night too, so it reproduces.
https://github.com/apache/lucene/actions/workflows/run-nightly-smoketester.yml

I looked it up to getAllLuceneReleases in the smoke tester script, which in
turn lists all releases available at:
https://archive.apache.org/dist/lucene/java/

version 9.10.1 isn't there so it complains with:
RuntimeError: tested version=9.10.1 but it was not released?

I'm not sure how to deal with yet-unreleased minor versions myself!

D.

On Thu, Apr 4, 2024 at 4:20 PM Benjamin Trent  wrote:

> This seems related to us forgetting to make the back-compat indices &
> versions when 9.10.1 was released and me adding them later.
>
> I have since added the 9.10.1 to Version.java and version.txt in main and
> 9x. Now, both main and 9x have the back-compat indices (these changes were
> not at the same time, and cause a separate build failure noticed by Mike
> M.).
>
> But this failure commit on main df154cdc2288a33747edb8849509ea5c3cbf792e,
> contains both of my changes (the 9.10.1 version & back compat indices).
>
> Was there something else forgotten during this bugfix release that we need
> to address? I am very new to the back-compat and release logic in Lucene
> and I am eager to learn.
>
> On Thu, Apr 4, 2024 at 9:43 AM Dawid Weiss  wrote:
>
>>
>> https://github.com/apache/lucene/actions/runs/8548297347/job/23421799032
>>
>> This smoketester run failed with:
>>
>> > RuntimeError: tested version=9.10.1 but it was not released?
>>
>> I guess it's not a hiccup but something recent?
>>
>> On Thu, Apr 4, 2024 at 3:24 AM Dawid Weiss 
>> wrote:
>>
>>>
>>> [image: GitHub] [apache/lucene] Run nightly: buildAndPushRelease and
>>> smokeTestRelease.py workflow run
>>>
>>>   Run nightly: buildAndPushRelease and smokeTestRelease.py: Some jobs
>>> were not successful
>>>
>>> View workflow run
>>> 
>>>
>>> [image: Smoke test release on jdk 21, ubuntu-latest]
>>>
>>> *Run nightly: buildAndPushRelease and smokeTestRelease.py* / Smoke test
>>> release on jdk 21, ubuntu-latest
>>> Failed in 9 minutes and 22 seconds
>>> [image: annotations for Run nightly: buildAndPushRelease and
>>> smokeTestRelease.py / Smoke test release on jdk 21, ubuntu-latest] 1
>>> 
>>> [image: Smoke test release on jdk 22-ea, ubuntu-latest]
>>>
>>> *Run nightly: buildAndPushRelease and smokeTestRelease.py* / Smoke test
>>> release on jdk 22-ea, ubuntu-latest
>>> Cancelled
>>> [image: annotations for Run nightly: buildAndPushRelease and
>>> smokeTestRelease.py / Smoke test release on jdk 22-ea, ubuntu-latest] 2
>>> 
>>>
>>>
>>>
>>> —
>>> You are receiving this because you are subscribed to this thread.
>>> Manage your GitHub Actions notifications
>>> 
>>>
>>>
>>> GitHub, Inc. ・88 Colin P Kelly Jr Street ・San Francisco, CA 94107
>>>
>>>
>>

Re: beasting tests

[Dawid Weiss]

> Thanks for the explanation. It makes sense that we start with a given
> seed and then each iteration is different because it re-uses the same
> Random instance (or whatever static state?) without re-initialization?
>

It doesn't reuse the same random instance - it's not that simple - it
hierarchically forks the random seed between tests, classes, reiterations
etc., and then initializes the context's random so that runs are
predictable and repeatable even if you run a subset of tests for the given
main seed. Random instances are also not reused between threads in an
attempt to make failures reproducible (this, of course, is not fool-proof).

If you care to look, there are more documented examples with somewhat
increasing complexity here:
https://github.com/randomizedtesting/randomizedtesting/tree/master/examples/maven/src/main/java/com/carrotsearch/examples/randomizedrunner

In particular, these talk about hierarchical seed propagation:

https://github.com/randomizedtesting/randomizedtesting/blob/master/examples/maven/src/main/java/com/carrotsearch/examples/randomizedrunner/Test004MoreRandomness.java
https://github.com/randomizedtesting/randomizedtesting/blob/master/examples/maven/src/main/java/com/carrotsearch/examples/randomizedrunner/Test006RepeatingTests.java

D.