Yeah thanks guys -- beast it is.
I wonder if we should not document tests.iters (a bit more expert), or add
a warning to it in the help output saying something like: NOTE: some tests
are incompatible because BeforeClass/AfterClass isn't performed inbetween.
Try beast.iters instead.
On Thu, Jan 11, 2018 at 8:39 PM Erick Erickson <erickerick...@gmail.com>
wrote:
> Ok, thanks both. That makes a lot of sense. I'll just use beasting for
> most anything SolrCloud related.
>
>
> On Thu, Jan 11, 2018 at 4:56 PM, Chris Hostetter <hossman_luc...@fucit.org
> > wrote:
>
>> : (I had left the comment in question)
>> : I think a test shouldn't have to explicitly clean up after itself,
>> except
>> : perhaps intra-method as-needed; test-infrastructure should do the class
>> : (test suite).
>>
>> All test code should always be expected to clean up their messes at
>> whatever "ending" stage corrisponds with the stage where the mess was
>> made.
>>
>> how the mess is cleaned up, and wether infrastructure/scaffolding code
>> helps do that dpeends on the specifics of the infrastucture/scaffolding in
>> question -- if you make a mess in a test method that the general purpose
>> infrastructure doesn't expect, then the burden is on you
>> to add the level of infrastructure (either in your specific test class, or
>> in a new abstract base class depending on how you think it might be
>> re-usable) to do so.
>>
>> In the abstract: Assume AbstractParentTest class that creates some
>> "parentMess" in @BeforeClass, and deletes "parentMess" in an
>> @AfterClass....
>>
>> 1) if you want all of your tests methods to have access to a
>> shiny new/unique instance of "childMess" in every test method, then burden
>> is on you to create/destroy childMess in your own @Before and @After
>> methods
>>
>> 2) If you want test methods that are going to mutate "parentMess" then the
>> burden is on you to ensure (ideally via @After methods that "do the right
>> thing" even if the test method fails) that "parentMess" is correctly reset
>> so that all the test methods depending on "parentMess" can run in any
>> order (or run multiple times in a single instance) ... either that, or you
>> shouldn't use AbstractParentTest -- you should create/destroy
>> a "parentMess" instance yourself in your @Before & @After methods
>>
>> Concretely...
>>
>> : > The assumption was that everything would be cleaned up between runs
>> : > doesn't appear to be true for SolrCloud tests. I think one of two
>> things is
>> : > happening:
>> : >
>> : > 1> collections (and perhaps aliases) are simply not cleaned up
>> : >
>> : > 2> there is a timing issue, we have waitForCollectionToDisappear in
>> test
>> : > code after all.
>>
>> ...these are vague statements ("everything", "SolrCloud tests", "not
>> cleaned up") and not being intimately familiar with the test class in
>> question it's not clear exactly is happening or what expectations various
>> people have -- BUT -- assuming this is in regards to
>> SolrCloudTestCase, that base class has very explicit docs about
>> how it's intended to be used: you are expected to configure & init a
>> MiniSolrCloudCluster instance in an @BeforeClass method -- it has helper
>> code for this -- and that cluster lives for the lifespan of the class at
>> which point an @AfterClass in SolrCloudTestCase will ensure it gets torn
>> down.
>>
>> Tests which subclass SolrCloudTestCase should be initializing the cluster
>> only in @BeforeClass. Most tests should only be creating collections in
>> @BeforeClass -- allthough you are certainly free to do things like
>> create/destroy collections on a per test method basis in @Before/@After
>> methods if you have a need for that sort of thing.
>>
>> If that's not the lifecycle you want -- if you want a lifecycle where ever
>> individual test method gets it's own pristine new MiniSolrCloudCluster
>> instance w/o any pre-existing collections, then you shouldn't use
>> SolrCloudTestCase -- you should just create/destroy
>> unique MiniSolrCloudCluster instances in your own @Before/@After methods.
>>
>>
>> Bottom Line: there is no one size fits all test scaffolding -- not when we
>> have some tests classes where we want to create a collection once, fill it
>> with lots of docs, and then re-use it in 100s of test methods, but other
>> classes want to test the very operation of creating/deleting collections.
>>
>> Use the tools that make sense for the test you're writting.
>>
>>
>>
>>
>> -Hoss
>> http://www.lucidworks.com/
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>>
>
--
Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
http://www.solrenterprisesearchserver.com
