Interesting! So, I changed my test.seed from 5B595A296E3623CE to 5B595A296E3623CC by changing the last letter from E to C, and reran the test, and it passed!
Be curious if it fails for anyone else? The failing seed version was complaining about various timeouts etc in dealing with the collections, is this worth opening a ticket about, or is it just something we live with? (Maybe that is the the thrust of your other thread!) > On Nov 22, 2021, at 10:11 AM, Gus Heck <gus.h...@gmail.com> wrote: > > The test.seed will cause the randomized testing to choose the same > locale/timezone/encoding/etc so that if you're hitting a bug that only shows > up with swahili or for the kiribati timezone it will reproduce reliably. > > This only works if your test is tripped by something influenced by that seed. > Otherwise: > > A) it's key to run cleanTest before running that "reproduce with" command so > that the build doesn't pass thinking the test is up to date (the reproduce > line probably ought to include this by default). > > B) The bad news is what you are probably seeing is flakey test issues. > Unfortunately, there is something wrong either with the test harness or the > server wherein at least two or maybe more things seem to happen. Test shut > down leaks threads relating to zkclient/httpclient, or collections that don't > seem to exist when they should. There may be more failure modes than that, > but those are two that I've noticed. These issues resolve when the tests are > run individually, and probably relate to something being slow due to the high > load of the tests. The thread leak version fails a test called "class method" > and the reproduce string always fails with a message about junit not finding > tests, because there isn't any test named .classMethod > > I recently started a thread titled "unit tests" where I so far have > documented how bad this is but have not yet found any solution. > > -Gus > > On Mon, Nov 22, 2021 at 7:30 AM Eric Pugh <ep...@opensourceconnections.com > <mailto:ep...@opensourceconnections.com>> wrote: > Can someone explain how the -Ptest.seed works, and how I use it to figure out > if a bug exists? > > I ran the full set of tests, and had this message: > ERROR: The following test(s) have failed: > - > org.apache.solr.handler.component.DistributedQueryComponentCustomSortTest.test > (:solr:core) > Test output: > /Users/epugh/Documents/projects/solr-epugh-2/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.handler.component.DistributedQueryComponentCustomSortTest.txt > Reproduce with: gradlew :solr:core:test --tests > "org.apache.solr.handler.component.DistributedQueryComponentCustomSortTest.test" > -Ptests.jvms=4 -Ptests.jvmargs=-XX:TieredStopAtLevel=1 > -Ptests.seed=5B595A296E3623CE -Ptests.file.encoding=US-ASCII > > > I then tried the same tests, but with out the -Ptests.seed, and it passed! > Does this mean there is a bug or not? > > I did go and check > > http://fucit.org/solr-jenkins-reports/history-trend-of-recent-failures.html#series/org.apache.solr.handler.component.DistributedQueryComponentCustomSortTest.test > > <http://fucit.org/solr-jenkins-reports/history-trend-of-recent-failures.html#series/org.apache.solr.handler.component.DistributedQueryComponentCustomSortTest.test> > > And it appears that this test has been failing more since July. > > Eric > > > > _______________________ > Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | > http://www.opensourceconnections.com <http://www.opensourceconnections.com/> > | My Free/Busy <http://tinyurl.com/eric-cal> > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed > <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw> > > This e-mail and all contents, including attachments, is considered to be > Company Confidential unless explicitly stated otherwise, regardless of > whether attachments are marked as such. > > > > -- > http://www.needhamsoftware.com <http://www.needhamsoftware.com/> (work) > http://www.the111shift.com <http://www.the111shift.com/> (play) _______________________ Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com <http://www.opensourceconnections.com/> | My Free/Busy <http://tinyurl.com/eric-cal> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw> This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.
