Re: [I] Improve smoketester JDK versions support [lucene]
dweiss commented on issue #13107: URL: https://github.com/apache/lucene/issues/13107#issuecomment-1948515415 So, here's what I managed to do so far. I've set up a simple gh workflow that can be triggered manually and provided with a release candidate to verify - here's what it looks like:  the workflow runs a matrix test against an array of JDKs: 11, 17, 21, 22-ea, automatically fetching the latest version of these (temurin distribution). A parameter allows one to turn off the tests (I'm not sure how long they'd take on github - was afraid to check...). Interestingly, I didn't have to touch the smoke tester at all as it already allows passing parameters to gradle, so I: * skipped all the tests by passing a filtering condition pointing at a non-existing test group, * compile and test against the matrix's java distribution using the existing runtime.java.home parameter we already have. I didn't modify the smoke tester so it reports that it runs gradle against Java 11, but reading the code I think it does the right thing and runs it against the provided Java version. Here is the whole workflow definition - https://raw.githubusercontent.com/dweiss/lucene/main/.github/workflows/run-smoketester.yml And here's a run I made to confirm it's working: https://github.com/dweiss/lucene/actions/runs/7932039311 note each run against a different JVM, including 22-ea, the logs confirming this:  The execution times are not bad at all - ~10 minutes per job and they're mostly in parallel, so no big deal I think. From here, I think it's a short walk to making a workflow that builds the distribution first, uploads it as an artifact which is then consumed by a job verifying the smoke tester against all the JVMs in question... unfortunately I won't be able to work on this the following week so if somebody wishes to take over and try, please go ahead. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
Re: [I] Improve smoketester JDK versions support [lucene]
uschindler commented on issue #13107: URL: https://github.com/apache/lucene/issues/13107#issuecomment-1946879030 @stefanvodita 's PR branch works on Policeman Jenkins (we had to disable the version check): https://jenkins.thetaphi.de/job/Lucene-Release-Tester-v2/3/console Once this succeeds I will give my +1 vote to Lucene 9.10.0 release :-) May take several hours. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
Re: [I] Improve smoketester JDK versions support [lucene]
stefanvodita commented on issue #13107: URL: https://github.com/apache/lucene/issues/13107#issuecomment-1946722027 > Can I pass multiple --test-alternative-java You can. For example, I ran: ``` JAVA_HOME=/usr/lib/jvm/java-11-amazon-corretto python3 -u dev-tools/scripts/smokeTestRelease.py --test-alternative-java /usr/lib/jvm/java-20-amazon-corretto --test-alternative-java /usr/lib/jvm/java-21-amazon-corretto \ https://dist.apache.org/repos/dist/dev/lucene/lucene-9.10.0-RC1-rev-695c0ac84508438302cd346a812cfa2fdc5a10df ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
Re: [I] Improve smoketester JDK versions support [lucene]
uschindler commented on issue #13107: URL: https://github.com/apache/lucene/issues/13107#issuecomment-1946709421 Thanks @stefanvodita. I will give it a try now on Policeman Jenkins! Maybe remove the hard coded java 17. I think instead of printing Java version it could just print the command line / JAVA_HOME. Can I pass multiple `--test-alternative-java` (I think this what `append` is for)? I think a better solution may save the version number also in the returned tuple, but for now this helps. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
Re: [I] Improve smoketester JDK versions support [lucene]
stefanvodita commented on issue #13107: URL: https://github.com/apache/lucene/issues/13107#issuecomment-1946685598 I was looking at this in the background and wrote a quick hack to add `--test-alternative-java` for branch_9x in #13108. I think it does what @uschindler wanted except print the JDK version numbers. I ran the script with JDK 20 and 21 and it seems to work. There's plenty we can do to make the script nicer, but I didn't have more time today. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
Re: [I] Improve smoketester JDK versions support [lucene]
uschindler commented on issue #13107: URL: https://github.com/apache/lucene/issues/13107#issuecomment-1946639414 Add a command line parameter to disable tests. The we could test it. Anyways: If you have a branch I can try it on policeman. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
Re: [I] Improve smoketester JDK versions support [lucene]
rmuir commented on issue #13107: URL: https://github.com/apache/lucene/issues/13107#issuecomment-1946634786 well, i think part of what makes changing the smoketester scary is that there's no validation of it in any PR I make. So if the github action would "verify smoketester works" but simply disable the execution of tests, it would make me feel a lot more comfortable. The tests will already be run by the other github action. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
Re: [I] Improve smoketester JDK versions support [lucene]
uschindler commented on issue #13107: URL: https://github.com/apache/lucene/issues/13107#issuecomment-1946620997 > I agree - I've just taken a look and tests consume a lot of time on gh jobs. The rest should be doable. It does not need to be Github (it would consume ASF's credits, which are limited). Jenkisn does the matrix testing already. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
Re: [I] Improve smoketester JDK versions support [lucene]
uschindler commented on issue #13107: URL: https://github.com/apache/lucene/issues/13107#issuecomment-1946614889 > There is value in people running smoke tester in various environments and configurations but I think we could also define a github workflow which would run the smoke tester against a matrix of JVMs (and maybe OSs). This is quite trivial to do and the benefit here is that these multiple jobs in the matrix run in parallel. > > The only thing I'm not sure of is whether a single check would fit within github's timeout limit (but I believe they should). I have a Jenkins Job for that. Running it in a matrix is possible, but does not help with the problem described here. The base JDK version (11 in 9.x) must always be provided and the alternative one can only Java 17. So I still would like to have the option to let Jenkins run with various JDKs in a flexible way. If you want to parallelize, you can do that - of course. But simplicity of setting up a Jenkins Job like I did is to run a single script with a suitable command line parameter. So basically we need smoketester to run with base Java (enforced to ensure we are compatible). The xtra JVM passed to run tests should be configureable. I'd still like to run all tests with the supplied Javas on the RELEASE.src file not from git checkout. So let's only fix the static spaghetti in smoker to run base and a number of extras. Uwe -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
Re: [I] Improve smoketester JDK versions support [lucene]
dweiss commented on issue #13107: URL: https://github.com/apache/lucene/issues/13107#issuecomment-1946607741 I agree - I've just taken a look and tests consume a lot of time on gh jobs. The rest should be doable. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
Re: [I] Improve smoketester JDK versions support [lucene]
rmuir commented on issue #13107: URL: https://github.com/apache/lucene/issues/13107#issuecomment-1946585638 You could even disable the running of tests (which might be the slowest part) for such a github workflow job. The real value is in the release checks and the packaging and all that, that isn't otherwise validated by any CI. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
Re: [I] Improve smoketester JDK versions support [lucene]
dweiss commented on issue #13107: URL: https://github.com/apache/lucene/issues/13107#issuecomment-1946558279 There is value in people running smoke tester in various environments and configurations but I think we could also define a github workflow which would run the smoke tester against a matrix of JVMs (and maybe OSs). This is quite trivial to do and the benefit here is that these multiple jobs in the matrix run in parallel. The only thing I'm not sure of is whether a single check would fit within github's timeout limit (but I believe they should). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
Re: [I] Improve smoketester JDK versions support [lucene]
rmuir commented on issue #13107: URL: https://github.com/apache/lucene/issues/13107#issuecomment-1946349998 yeah, i think for the smoketester we should just test only java versions supported by gradle? Since this is different than just running tests but trying to check that much more stuff works correctly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
Re: [I] Improve smoketester JDK versions support [lucene]
uschindler commented on issue #13107: URL: https://github.com/apache/lucene/issues/13107#issuecomment-1946346647 > gradle should fail with a clear error though if the version is not supported. Correct. My problem is in addition: I want to also test java 22, but Gradle refuses to run this one, you need RUNTIME_JAVA_HOME and execute with earlier one. Maybe I run test tests at command line then. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
Re: [I] Improve smoketester JDK versions support [lucene]
rmuir commented on issue #13107: URL: https://github.com/apache/lucene/issues/13107#issuecomment-1946308313 gradle should fail with a clear error though if the version is not supported. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
Re: [I] Improve smoketester JDK versions support [lucene]
uschindler commented on issue #13107: URL: https://github.com/apache/lucene/issues/13107#issuecomment-1946306542 I think it may still parse the Java version passed as alternative java homes, but just ensure they are >= the base version. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
