Any test that doesn't really do anything in setup/teardown seems like a
good candidate for using tests.iters, which will be much faster.
For SolrCloud level tests beasting is becoming my first choice.
For a lot of the Lucene level code tests.iters makes a lot of sense so
it's too useful to remove.
Best,
Erick
On Mon, Jan 15, 2018 at 7:56 AM, Jason Gerlowski <gerlowsk...@gmail.com> wrote:
> It sounds like the recommendation in this thread is to _always_ use
> "ant beast" instead of "tests.iters". Is there _any_ case where
> "tests.iters" should be preferred? If not, should we remove support
> for "tests.iters" to remove any ambiguity? (Especially since this has
> come up on the list a few times...)
>
> On Fri, Jan 12, 2018 at 11:14 AM, Erick Erickson
> <erickerick...@gmail.com> wrote:
>> Well, since I'm in there anyway I'll include the note in the patch. At
>> least that'll alert people to dig deeper.
>>
>> On Thu, Jan 11, 2018 at 8:34 PM, David Smiley <david.w.smi...@gmail.com>
>> wrote:
>>> Yeah thanks guys -- beast it is.
>>>
>>> I wonder if we should not document tests.iters (a bit more expert), or add a
>>> warning to it in the help output saying something like: NOTE: some tests are
>>> incompatible because BeforeClass/AfterClass isn't performed inbetween. Try
>>> beast.iters instead.
>>>
>>> On Thu, Jan 11, 2018 at 8:39 PM Erick Erickson <erickerick...@gmail.com>
>>> wrote:
>>>>
>>>> Ok, thanks both. That makes a lot of sense. I'll just use beasting for
>>>> most anything SolrCloud related.
>>>>
>>>>
>>>> On Thu, Jan 11, 2018 at 4:56 PM, Chris Hostetter
>>>> <hossman_luc...@fucit.org> wrote:
>>>>>
>>>>> : (I had left the comment in question)
>>>>> : I think a test shouldn't have to explicitly clean up after itself,
>>>>> except
>>>>> : perhaps intra-method as-needed; test-infrastructure should do the class
>>>>> : (test suite).
>>>>>
>>>>> All test code should always be expected to clean up their messes at
>>>>> whatever "ending" stage corrisponds with the stage where the mess was
>>>>> made.
>>>>>
>>>>> how the mess is cleaned up, and wether infrastructure/scaffolding code
>>>>> helps do that dpeends on the specifics of the infrastucture/scaffolding
>>>>> in
>>>>> question -- if you make a mess in a test method that the general purpose
>>>>> infrastructure doesn't expect, then the burden is on you
>>>>> to add the level of infrastructure (either in your specific test class,
>>>>> or
>>>>> in a new abstract base class depending on how you think it might be
>>>>> re-usable) to do so.
>>>>>
>>>>> In the abstract: Assume AbstractParentTest class that creates some
>>>>> "parentMess" in @BeforeClass, and deletes "parentMess" in an
>>>>> @AfterClass....
>>>>>
>>>>> 1) if you want all of your tests methods to have access to a
>>>>> shiny new/unique instance of "childMess" in every test method, then
>>>>> burden
>>>>> is on you to create/destroy childMess in your own @Before and @After
>>>>> methods
>>>>>
>>>>> 2) If you want test methods that are going to mutate "parentMess" then
>>>>> the
>>>>> burden is on you to ensure (ideally via @After methods that "do the right
>>>>> thing" even if the test method fails) that "parentMess" is correctly
>>>>> reset
>>>>> so that all the test methods depending on "parentMess" can run in any
>>>>> order (or run multiple times in a single instance) ... either that, or
>>>>> you
>>>>> shouldn't use AbstractParentTest -- you should create/destroy
>>>>> a "parentMess" instance yourself in your @Before & @After methods
>>>>>
>>>>> Concretely...
>>>>>
>>>>> : > The assumption was that everything would be cleaned up between runs
>>>>> : > doesn't appear to be true for SolrCloud tests. I think one of two
>>>>> things is
>>>>> : > happening:
>>>>> : >
>>>>> : > 1> collections (and perhaps aliases) are simply not cleaned up
>>>>> : >
>>>>> : > 2> there is a timing issue, we have waitForCollectionToDisappear in
>>>>> test
>>>>> : > code after all.
>>>>>
>>>>> ...these are vague statements ("everything", "SolrCloud tests", "not
>>>>> cleaned up") and not being intimately familiar with the test class in
>>>>> question it's not clear exactly is happening or what expectations various
>>>>> people have -- BUT -- assuming this is in regards to
>>>>> SolrCloudTestCase, that base class has very explicit docs about
>>>>> how it's intended to be used: you are expected to configure & init a
>>>>> MiniSolrCloudCluster instance in an @BeforeClass method -- it has helper
>>>>> code for this -- and that cluster lives for the lifespan of the class at
>>>>> which point an @AfterClass in SolrCloudTestCase will ensure it gets torn
>>>>> down.
>>>>>
>>>>> Tests which subclass SolrCloudTestCase should be initializing the cluster
>>>>> only in @BeforeClass. Most tests should only be creating collections in
>>>>> @BeforeClass -- allthough you are certainly free to do things like
>>>>> create/destroy collections on a per test method basis in @Before/@After
>>>>> methods if you have a need for that sort of thing.
>>>>>
>>>>> If that's not the lifecycle you want -- if you want a lifecycle where
>>>>> ever
>>>>> individual test method gets it's own pristine new MiniSolrCloudCluster
>>>>> instance w/o any pre-existing collections, then you shouldn't use
>>>>> SolrCloudTestCase -- you should just create/destroy
>>>>> unique MiniSolrCloudCluster instances in your own @Before/@After methods.
>>>>>
>>>>>
>>>>> Bottom Line: there is no one size fits all test scaffolding -- not when
>>>>> we
>>>>> have some tests classes where we want to create a collection once, fill
>>>>> it
>>>>> with lots of docs, and then re-use it in 100s of test methods, but other
>>>>> classes want to test the very operation of creating/deleting collections.
>>>>>
>>>>> Use the tools that make sense for the test you're writting.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> -Hoss
>>>>> http://www.lucidworks.com/
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>>>>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>>>>
>>>>
>>>
>>>
>>> --
>>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>>> LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
>>> http://www.solrenterprisesearchserver.com
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
