Hi Dawid,
cool thanks for the pointer. I really like this list 😊 So I am not sure at which point we ever had randomization of security manager and/or asserts. I assume, Policeman Jenkins never had that. I have the feeling that Elastic did this on their build servers, but that’s also not proved. I suggested that change in one of my talks at BerlinBuzzwords, but may have never implemented it. Anyways: I am open to add randomization on Jenkins, it’s just 2 lines of code in the randomize-java-groovy file on Policeman Jenkins. Maybe disable asserts/and or SecurityManager in 1/5th of all cases. Uwe ----- Uwe Schindler Achterdiek 19, D-28357 Bremen https://www.thetaphi.de eMail: u...@thetaphi.de From: Dawid Weiss <dawid.we...@gmail.com> Sent: Friday, February 19, 2021 7:52 PM To: Lucene Dev <dev@lucene.apache.org> Subject: Re: Random disabling of asserts in tests is not working Hi Uwe, No, it's not randomized - always runs with the security manager enabled. All the options are here: https://github.com/apache/lucene-solr/blob/master/gradle/testing/randomization.gradle#L68-L103 When the value says "random" we pick the random value at runtime (so that it also works within IDEs). We could pick security manager at build-time (derive from project seed). This is a no-brainer to do. As Robert said - perhaps we should keep some things more strict for developers and just shuffle on the CI-only. This requires passing -Ptests.*=... flags but is simple, I think. Dawid On Fri, Feb 19, 2021 at 7:45 PM Uwe Schindler <u...@thetaphi.de <mailto:u...@thetaphi.de> > wrote: Hi, I don’t fully remember what the setup previously was, but at least for master and 8.x it does not automatically enable/disable asserts. We can of course do this together with the other settings like GC or compressed OOPs, its just a few more lines in the Groovy file. I was also thinking that we have Security Manager enabled/disabled from time to time. But recently, I see no randomization for this on Jenkins, unless it’s part of the Gradle build. Uwe ----- Uwe Schindler Achterdiek 19, D-28357 Bremen https://www.thetaphi.de eMail: u...@thetaphi.de <mailto:u...@thetaphi.de> From: Robert Muir <rcm...@gmail.com <mailto:rcm...@gmail.com> > Sent: Friday, February 19, 2021 3:13 PM To: dev@lucene.apache.org <mailto:dev@lucene.apache.org> Subject: Re: Random disabling of asserts in tests is not working I don't think it is enabled (at least in policeman jenkins). perhaps it didn't work correctly when the build was cutover to gradle. Take a look at any old build such as https://jenkins.thetaphi.de/view/Lucene-Solr/job/Lucene-Solr-master-Linux/29491/ . You can see the variables it randomizes right there. You can confirm by clicking console->full log and it prints exact gradle command that it runs: https://jenkins.thetaphi.de/view/Lucene-Solr/job/Lucene-Solr-master-Linux/29491/consoleFull Let's look into it, in a couple weeks or so? On Fri, Feb 19, 2021 at 8:32 AM Michael McCandless <luc...@mikemccandless.com <mailto:luc...@mikemccandless.com> > wrote: On Fri, Feb 19, 2021 at 8:07 AM Robert Muir <rcm...@gmail.com <mailto:rcm...@gmail.com> > wrote: I think it has a downside: having a bug in an assert is really more of a corner case. This is the kind of thing jenkins is for? Ahh, that is indeed a really good point. I would want/expect asserts to always work correctly when running local tests ... if we randomly disabled them in our checkouts it can cause a false sense of security, too soon. OK, I agree, let's leave it as randomization in Jenkins! How do we know that Jenkins job/s are still randomizing assertions? Who tests the tester? Mike McCandless http://blog.mikemccandless.com
