It sounds like the recommendation in this thread is to _always_ use
"ant beast" instead of "tests.iters". Is there _any_ case where
"tests.iters" should be preferred? If not, should we remove support
for "tests.iters" to remove any ambiguity? (Especially since this has
come up on the list a few times...)
On Fri, Jan 12, 2018 at 11:14 AM, Erick Erickson
<erickerick...@gmail.com> wrote:
> Well, since I'm in there anyway I'll include the note in the patch. At
> least that'll alert people to dig deeper.
>
> On Thu, Jan 11, 2018 at 8:34 PM, David Smiley <david.w.smi...@gmail.com>
> wrote:
>> Yeah thanks guys -- beast it is.
>>
>> I wonder if we should not document tests.iters (a bit more expert), or add a
>> warning to it in the help output saying something like: NOTE: some tests are
>> incompatible because BeforeClass/AfterClass isn't performed inbetween. Try
>> beast.iters instead.
>>
>> On Thu, Jan 11, 2018 at 8:39 PM Erick Erickson <erickerick...@gmail.com>
>> wrote:
>>>
>>> Ok, thanks both. That makes a lot of sense. I'll just use beasting for
>>> most anything SolrCloud related.
>>>
>>>
>>> On Thu, Jan 11, 2018 at 4:56 PM, Chris Hostetter
>>> <hossman_luc...@fucit.org> wrote:
>>>>
>>>> : (I had left the comment in question)
>>>> : I think a test shouldn't have to explicitly clean up after itself,
>>>> except
>>>> : perhaps intra-method as-needed; test-infrastructure should do the class
>>>> : (test suite).
>>>>
>>>> All test code should always be expected to clean up their messes at
>>>> whatever "ending" stage corrisponds with the stage where the mess was
>>>> made.
>>>>
>>>> how the mess is cleaned up, and wether infrastructure/scaffolding code
>>>> helps do that dpeends on the specifics of the infrastucture/scaffolding
>>>> in
>>>> question -- if you make a mess in a test method that the general purpose
>>>> infrastructure doesn't expect, then the burden is on you
>>>> to add the level of infrastructure (either in your specific test class,
>>>> or
>>>> in a new abstract base class depending on how you think it might be
>>>> re-usable) to do so.
>>>>
>>>> In the abstract: Assume AbstractParentTest class that creates some
>>>> "parentMess" in @BeforeClass, and deletes "parentMess" in an
>>>> @AfterClass....
>>>>
>>>> 1) if you want all of your tests methods to have access to a
>>>> shiny new/unique instance of "childMess" in every test method, then
>>>> burden
>>>> is on you to create/destroy childMess in your own @Before and @After
>>>> methods
>>>>
>>>> 2) If you want test methods that are going to mutate "parentMess" then
>>>> the
>>>> burden is on you to ensure (ideally via @After methods that "do the right
>>>> thing" even if the test method fails) that "parentMess" is correctly
>>>> reset
>>>> so that all the test methods depending on "parentMess" can run in any
>>>> order (or run multiple times in a single instance) ... either that, or
>>>> you
>>>> shouldn't use AbstractParentTest -- you should create/destroy
>>>> a "parentMess" instance yourself in your @Before & @After methods
>>>>
>>>> Concretely...
>>>>
>>>> : > The assumption was that everything would be cleaned up between runs
>>>> : > doesn't appear to be true for SolrCloud tests. I think one of two
>>>> things is
>>>> : > happening:
>>>> : >
>>>> : > 1> collections (and perhaps aliases) are simply not cleaned up
>>>> : >
>>>> : > 2> there is a timing issue, we have waitForCollectionToDisappear in
>>>> test
>>>> : > code after all.
>>>>
>>>> ...these are vague statements ("everything", "SolrCloud tests", "not
>>>> cleaned up") and not being intimately familiar with the test class in
>>>> question it's not clear exactly is happening or what expectations various
>>>> people have -- BUT -- assuming this is in regards to
>>>> SolrCloudTestCase, that base class has very explicit docs about
>>>> how it's intended to be used: you are expected to configure & init a
>>>> MiniSolrCloudCluster instance in an @BeforeClass method -- it has helper
>>>> code for this -- and that cluster lives for the lifespan of the class at
>>>> which point an @AfterClass in SolrCloudTestCase will ensure it gets torn
>>>> down.
>>>>
>>>> Tests which subclass SolrCloudTestCase should be initializing the cluster
>>>> only in @BeforeClass. Most tests should only be creating collections in
>>>> @BeforeClass -- allthough you are certainly free to do things like
>>>> create/destroy collections on a per test method basis in @Before/@After
>>>> methods if you have a need for that sort of thing.
>>>>
>>>> If that's not the lifecycle you want -- if you want a lifecycle where
>>>> ever
>>>> individual test method gets it's own pristine new MiniSolrCloudCluster
>>>> instance w/o any pre-existing collections, then you shouldn't use
>>>> SolrCloudTestCase -- you should just create/destroy
>>>> unique MiniSolrCloudCluster instances in your own @Before/@After methods.
>>>>
>>>>
>>>> Bottom Line: there is no one size fits all test scaffolding -- not when
>>>> we
>>>> have some tests classes where we want to create a collection once, fill
>>>> it
>>>> with lots of docs, and then re-use it in 100s of test methods, but other
>>>> classes want to test the very operation of creating/deleting collections.
>>>>
>>>> Use the tools that make sense for the test you're writting.
>>>>
>>>>
>>>>
>>>>
>>>> -Hoss
>>>> http://www.lucidworks.com/
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>>>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>>>
>>>
>>
>>
>> --
>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>> LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
>> http://www.solrenterprisesearchserver.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
