On Wed, Apr 21, 2010 at 1:37 PM, Stéphane Ducasse <[email protected] > wrote:
> > On Apr 21, 2010, at 11:47 AM, Peter Hugosson-Miller wrote: > > > On Wed, Apr 21, 2010 at 11:34 AM, Tudor Girba <[email protected]> > wrote: > > Basically yes. > > > > It's not just for giving it to someone else, but also for yourself. The > idea is to quickly notice when tests change color (from green to red), so > instead of remembering how many failed before and comparing with the current > number of failures, you simply mark them as expected failures and your bar > becomes green. > > > > Thanks, I think I get it now. > > > > So really, I shouldn't expect to see any expected failures in Pharo 1.0, > but since Pharo 1.1 is still in alpha, it would be OK to have some there > until it's ready for release. > > in a perfect world now in pharo we have tests that are red and we could not > fixed easily. > Ah, OK. I guess it's really a question of perception. The world in which I work is a bit different from the wonderful world of Pharo (no sarcasm intended!), in that we can't ship *anything* unless *all* our tests are green. The concept of "expected failures" just has no place, because as long as there are failures we still have work to do. I can understand Doru's description of an expected failure as a kind of bookmark, to be added and discarded during the day, but I get a bit nervous that the sight of too much green might make me miss the fact that I'm not finished yet :-p > -- > > Cheers, > > Peter > > > > Doru > > > > > > > > On 21 Apr 2010, at 11:28, Peter Hugosson-Miller wrote: > > > > Thanks for answering, Doru! > > > > So if I've understood you correctly, expected failures are useful when > one wants to give some code to someone else, with all the tests running > green, but at the same time let that person know that some specific tests > are really still red, but known about. In other words, are they simply a way > of documenting "work in progress", and not for production code? > > > > -- > > Cheers, > > Peter > > > > On Wed, Apr 21, 2010 at 11:12 AM, Tudor Girba <[email protected]> > wrote: > > Hi, > > > > The idea behind this is that you can use unit tests to document things > you have not done yet, or bugs that you know about but you do not want to > spend time working on right now. > > > > Simply reverting the assertion in your test does make the test runner > green, but it fails to document the intention, and a newcomer might get to > the false conclusion that the answer is not 42. By marking the test as > expected failure, you make the test runner green, but you also explicitly > say that the answer should be 42, but the machine is not quite perfect yet. > > > > Cheers, > > Doru > > > > > > > > On 21 Apr 2010, at 11:06, Peter Hugosson-Miller wrote: > > > > I'm trying to follow along with this discussion, but now I feel really > stupid :-( > > > > I've been using SUnit since 1998 (when it was still called > "BeckTestingFramework"), and ever since "expected failures" showed up, I've > never understood them. So I wonder if some kind guru-like person could > please explain to me what they are useful for? I mean, to my way of > thinking, if one writes a test that is expected to fail, then why not invert > the test and call it a success instead? > > > > for example: > > > > self should: [answer = 42] > > > > ...as an expected failure could simply be re-written as > > > > self should: [answer ~= 42] > > > > ...right? No, obviously I've missed something really obvious and > important, and that's why I'm asking the question now. Please be gentle ;-) > > > > -- > > Cheers, > > Peter > > > > On Wed, Apr 21, 2010 at 10:30 AM, Stéphane Ducasse < > [email protected]> wrote: > > We should have a look at what adrian did now the problem is that > understanding a large set of changes is more difficult than a couple of > simple ones. > > If somebody want to help we are open. > > Stef > > > > > > > I think Adrian Kuhn did that in his SUnit work. I also remember he also > introduced a difference between expectedFailures and expectedErrors. > > > > > > Doru > > > > > > > > > On 21 Apr 2010, at 10:16, Stéphane Ducasse wrote: > > > > > >> > > >> On Apr 21, 2010, at 9:51 AM, Adrian Lienhard wrote: > > >> > > >>> Yea, I agree, the GUI is suboptimal. > > >>> > > >>> I still think, though, that treating this case as a failure is > correct. For instance, consider the case where you had added a workaround to > a known bug and when the bug is fixed you need to remove the workaround > again. Maybe it even leads to a wrong behavior now that the bug is gone. In > this case you really want to know that the test does not fail anymore. > > >> > > >> yes > > >> Now I have the impression that expectedFailures should be like passes, > failed, errors: a state of the tests. > > >> > > >> Stef > > >> > > >>> In any case, I think that tagging methods as expected failures should > be done with pragmas and not with #expectedFailures. Like this it would also > be much easier to understand what's going on when you have a failure in this > test although all assertions pass. > > >>> > > >>> Adrian > > >>> > > >>> On Apr 21, 2010, at 08:22 , Stéphane Ducasse wrote: > > >>> > > >>>> > > >>>> On Apr 20, 2010, at 11:20 PM, Adrian Lienhard wrote: > > >>>> > > >>>>> Yes, if a test that is expected to fail does not fail, this is > treated as a failure. I think that makes sense. > > >>>> > > >>>> well it depends about the scenario. > > >>>> you put on expectedfailures something that gets in your way now, so > after if it works even better. > > >>>> of course you should get notified that the test is green while > expected it to failed. > > >>>> > > >>>> Now it leads to a UI problem where you have a failure that passes so > when you click on it nothing happens: no debugger. > > >>>> And you can wonder why the hell do I have a failure when my tests > pass. > > >>>> > > >>>> So I think that this implementation of expectedFailures is a hack. > > >>>> > > >>>>> > > >>>>> Adrian > > >>>>> > > >>>>> On Apr 20, 2010, at 21:57 , Stéphane Ducasse wrote: > > >>>>> > > >>>>>> Hi > > >>>>>> > > >>>>>> I tagged some tests as expected failures and I got a strange > behavior. > > >>>>>> On the the tests which was passing was listed under the failures. > > >>>>>> When I renamed the method without updating the expected failures > my bar was green. > > >>>>>> So expected failures really expect that the tests failed? We > cannto have green tests in there? > > >>>>>> > > >>>>>> Stef > > >>>>>> _______________________________________________ > > >>>>>> Pharo-project mailing list > > >>>>>> [email protected] > > >>>>>> > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > >>>>> > > >>>>> > > >>>>> _______________________________________________ > > >>>>> Pharo-project mailing list > > >>>>> [email protected] > > >>>>> > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > >>>> > > >>>> > > >>>> _______________________________________________ > > >>>> Pharo-project mailing list > > >>>> [email protected] > > >>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > >>> > > >>> > > >>> _______________________________________________ > > >>> Pharo-project mailing list > > >>> [email protected] > > >>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > >> > > >> > > >> _______________________________________________ > > >> Pharo-project mailing list > > >> [email protected] > > >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > > > > > -- > > > www.tudorgirba.com > > > > > > "Live like you mean it." > > > > > > > > > _______________________________________________ > > > Pharo-project mailing list > > > [email protected] > > > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > > > > > _______________________________________________ > > Pharo-project mailing list > > [email protected] > > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > > > > > _______________________________________________ > > Pharo-project mailing list > > [email protected] > > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > > > -- > > www.tudorgirba.com > > > > "Reasonable is what we are accustomed with." > > > > > > > > _______________________________________________ > > Pharo-project mailing list > > [email protected] > > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > > > _______________________________________________ > > Pharo-project mailing list > > [email protected] > > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > > > -- > > www.tudorgirba.com > > > > "Beauty is where we see it." > > > > > > > > > > > > _______________________________________________ > > Pharo-project mailing list > > [email protected] > > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > > > > > _______________________________________________ > > Pharo-project mailing list > > [email protected] > > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > > _______________________________________________ > Pharo-project mailing list > [email protected] > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > -- Cheers, Peter
_______________________________________________ Pharo-project mailing list [email protected] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
