Well, since I'm in there anyway I'll include the note in the patch. At
least that'll alert people to dig deeper.
On Thu, Jan 11, 2018 at 8:34 PM, David Smiley <david.w.smi...@gmail.com> wrote:
> Yeah thanks guys -- beast it is.
>
> I wonder if we should not document tests.iters (a bit more expert), or add a
> warning to it in the help output saying something like: NOTE: some tests are
> incompatible because BeforeClass/AfterClass isn't performed inbetween. Try
> beast.iters instead.
>
> On Thu, Jan 11, 2018 at 8:39 PM Erick Erickson <erickerick...@gmail.com>
> wrote:
>>
>> Ok, thanks both. That makes a lot of sense. I'll just use beasting for
>> most anything SolrCloud related.
>>
>>
>> On Thu, Jan 11, 2018 at 4:56 PM, Chris Hostetter
>> <hossman_luc...@fucit.org> wrote:
>>>
>>> : (I had left the comment in question)
>>> : I think a test shouldn't have to explicitly clean up after itself,
>>> except
>>> : perhaps intra-method as-needed; test-infrastructure should do the class
>>> : (test suite).
>>>
>>> All test code should always be expected to clean up their messes at
>>> whatever "ending" stage corrisponds with the stage where the mess was
>>> made.
>>>
>>> how the mess is cleaned up, and wether infrastructure/scaffolding code
>>> helps do that dpeends on the specifics of the infrastucture/scaffolding
>>> in
>>> question -- if you make a mess in a test method that the general purpose
>>> infrastructure doesn't expect, then the burden is on you
>>> to add the level of infrastructure (either in your specific test class,
>>> or
>>> in a new abstract base class depending on how you think it might be
>>> re-usable) to do so.
>>>
>>> In the abstract: Assume AbstractParentTest class that creates some
>>> "parentMess" in @BeforeClass, and deletes "parentMess" in an
>>> @AfterClass....
>>>
>>> 1) if you want all of your tests methods to have access to a
>>> shiny new/unique instance of "childMess" in every test method, then
>>> burden
>>> is on you to create/destroy childMess in your own @Before and @After
>>> methods
>>>
>>> 2) If you want test methods that are going to mutate "parentMess" then
>>> the
>>> burden is on you to ensure (ideally via @After methods that "do the right
>>> thing" even if the test method fails) that "parentMess" is correctly
>>> reset
>>> so that all the test methods depending on "parentMess" can run in any
>>> order (or run multiple times in a single instance) ... either that, or
>>> you
>>> shouldn't use AbstractParentTest -- you should create/destroy
>>> a "parentMess" instance yourself in your @Before & @After methods
>>>
>>> Concretely...
>>>
>>> : > The assumption was that everything would be cleaned up between runs
>>> : > doesn't appear to be true for SolrCloud tests. I think one of two
>>> things is
>>> : > happening:
>>> : >
>>> : > 1> collections (and perhaps aliases) are simply not cleaned up
>>> : >
>>> : > 2> there is a timing issue, we have waitForCollectionToDisappear in
>>> test
>>> : > code after all.
>>>
>>> ...these are vague statements ("everything", "SolrCloud tests", "not
>>> cleaned up") and not being intimately familiar with the test class in
>>> question it's not clear exactly is happening or what expectations various
>>> people have -- BUT -- assuming this is in regards to
>>> SolrCloudTestCase, that base class has very explicit docs about
>>> how it's intended to be used: you are expected to configure & init a
>>> MiniSolrCloudCluster instance in an @BeforeClass method -- it has helper
>>> code for this -- and that cluster lives for the lifespan of the class at
>>> which point an @AfterClass in SolrCloudTestCase will ensure it gets torn
>>> down.
>>>
>>> Tests which subclass SolrCloudTestCase should be initializing the cluster
>>> only in @BeforeClass. Most tests should only be creating collections in
>>> @BeforeClass -- allthough you are certainly free to do things like
>>> create/destroy collections on a per test method basis in @Before/@After
>>> methods if you have a need for that sort of thing.
>>>
>>> If that's not the lifecycle you want -- if you want a lifecycle where
>>> ever
>>> individual test method gets it's own pristine new MiniSolrCloudCluster
>>> instance w/o any pre-existing collections, then you shouldn't use
>>> SolrCloudTestCase -- you should just create/destroy
>>> unique MiniSolrCloudCluster instances in your own @Before/@After methods.
>>>
>>>
>>> Bottom Line: there is no one size fits all test scaffolding -- not when
>>> we
>>> have some tests classes where we want to create a collection once, fill
>>> it
>>> with lots of docs, and then re-use it in 100s of test methods, but other
>>> classes want to test the very operation of creating/deleting collections.
>>>
>>> Use the tools that make sense for the test you're writting.
>>>
>>>
>>>
>>>
>>> -Hoss
>>> http://www.lucidworks.com/
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>>
>>
>
>
> --
> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
> LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
> http://www.solrenterprisesearchserver.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
