Maybe there just needs to be better documentation about running unit tests then.
I have one argument for having unit tests with the gem - if i install a gem then i usually check if it has any tests with it and if there isn't any tests then i will just delete the gem, no questions asked. I'm not going to search where the source code itself is located or anything like that. If there's no tests within the gem then it has to be a bad gem - this is what i'm thinking. That's why i would like to have tests with the gems. Even if there's going to be any unified API or not. It shouldn't be harder to run tests than to actually use the lib thus having tests at some separate place and setting them (separately) up isn't good option either. Also, why do you think that Watirspec is a subset of Watir/FireWatir tests? I can see it as a pretty good replacement of current tests. Yeah, there are few functionalities which aren't tested by Watirspec like #attach and #click_no_wait, but other than that i'd say the suite is pretty decent. Or has someone made an explicit analysis of Watirspec being a superset rather than just guessing? Jarmo 2010/9/28 Bret Pettichord <[email protected]>: > There is another, separate reason, why I am in favor of removing the unit > tests from the gem. > > We used to do this, but we would often get complaints from users who > couldn't run the unit tests. The problem is that there are a couple of > additional requirements that you need to meet in order to get the unit tests > working. Users tend to treat the unit tests as a way of testing whether they > have installed Watir correctly, but this isn't what it is designed to do nor > accomplishes. So instead, they run into problems with the unit tests and > think that these are problems they need to get sorted out. But they aren't. > > One of these issues is that you have to turn off one of the security > settings in IE before running the tests. We also saw a recent thread about > UAC access complicating unit tests. > > I think what people want is just something that verifies whether Watir is > installed correctly. Maybe we could create something that does this and > package it as a unit test and then separately maintain the old "unit tests" > as a compatability/regression suite separately. > > Bret > > 2010/9/27 Bret Pettichord <[email protected]> >> >> Right they used to be separate. Actually they use to be one test suite, >> but then were duplicated when firewatir was forked from watir. Now they >> contain lots of duplication, but also lots of subtle differences. >> >> I was trying to remove duplication. Also I added an option to each test >> suite that allowed it to be run against the other implementation. This >> "compatibility" mode was helpful in finding inconsistencies between the two >> implementations. These features required adding interdependencies that broke >> the ability to run in a gem. My goal was to eventually migrate all of the >> tests out into a separate project that could be used to run against any >> implementation. This turned out to be a bigger task than I expected because >> of the large number of minor changes that were made, separately, in each of >> the two test suites. >> >> Yes Jari's watirspec is something like a unified test suite, but it is a >> subset of what we have in the watir/firewatir project. >> >> I think it is interesting to discuss what to do about this, but this has >> been a known issue for some two years and probably not something that can >> reasonably be resolved in the time frame of the 1.6.6 release. This is why I >> suggest we simply remove the unit tests from this version of the gem. >> >> Bret >> >> On Mon, Sep 27, 2010 at 2:42 PM, Jarmo <[email protected]> wrote: >>> >>> Isn't this completely separate problem? >>> >>> I see that currently there are unittests under FireWatir and unittests >>> under Watir. They should be executable without depending of eachother, >>> thus should be executable within installed gem and not just from the >>> source. I would try to solve that problem first without creating any >>> unified tests. >>> >>> When talking about unified Watir-API test suite, then isn't Jari's >>> Watirspec something like that? >>> >>> Again, for me these things seem to be different problems. Correct me >>> if i'm wrong. >>> >>> Jarmo >>> >>> On Mon, Sep 27, 2010 at 10:38 PM, Bret Pettichord <[email protected]> >>> wrote: >>> > The reason we have this problem is that what we really need is a >>> > unified >>> > Watir-API test suite that could be run against any implementation. >>> > >>> > I was trying to convert our existing unit tests into such a test suite, >>> > but >>> > ran out of time to complete this work and thus it is in an interim >>> > state. >>> > >>> > One way to fix the problem would be to roll back this work. If this is >>> > your >>> > plan, then i think it is a bad idea. >>> > >>> > Another way to fix the problem would be to complete this work and >>> > migrate >>> > all of the "unit tests" out of the individual gems. If this is your >>> > plan, >>> > then it is a good idea, but it also will probably result in not having >>> > the >>> > "unit tests" inside the individual gems. >>> > >>> > So I guess the question is, what is your plan? >>> > >>> > Bret >>> > >>> > On Mon, Sep 27, 2010 at 2:18 PM, Jarmo <[email protected]> wrote: >>> >> >>> >> Why not fix it? If i say that i have plan to do it, is that a bad >>> >> plan? >>> >> Why? >>> >> >>> >> Jarmo >>> >> >>> >> On Mon, Sep 27, 2010 at 9:08 PM, Bret Pettichord <[email protected]> >>> >> wrote: >>> >> > Yes the problem running unit tests from the gems is not fixed and >>> >> > there >>> >> > is >>> >> > no plan to fix it. >>> >> > >>> >> > It would be best not to ship the unit tests with the gems. >>> >> > >>> >> > If we want to run the unit tests on different platforms, we should >>> >> > do >>> >> > this >>> >> > by checking out the source from github, not installing the gems. >>> >> > >>> >> > Bret >>> >> > >>> >> > On Mon, Sep 27, 2010 at 7:34 AM, Željko Filipin >>> >> > <[email protected]> wrote: >>> >> >> >>> >> >> On Fri, Sep 24, 2010 at 11:04 PM, Charley Baker >>> >> >> <[email protected]> >>> >> >> wrote: >>> >> >> > we can take any >>> >> >> > help from documentation to running tests on various OSes. >>> >> >> >>> >> >> I can run unit tests on all major OSes. Is it still the case that >>> >> >> there >>> >> >> is >>> >> >> something wrong with unit tests provided with the gem? >>> >> >> >>> >> >> Željko >>> >> >> >>> >> >> _______________________________________________ >>> >> >> Wtr-development mailing list >>> >> >> [email protected] >>> >> >> http://rubyforge.org/mailman/listinfo/wtr-development >>> >> > >>> >> > >>> >> > >>> >> > -- >>> >> > Bret Pettichord >>> >> > Lead Developer, Watir, www.watir.com >>> >> > >>> >> > Blog, www.io.com/~wazmo/blog >>> >> > Twitter, www.twitter.com/bpettichord >>> >> > >>> >> > >>> >> > _______________________________________________ >>> >> > Wtr-development mailing list >>> >> > [email protected] >>> >> > http://rubyforge.org/mailman/listinfo/wtr-development >>> >> > >>> >> _______________________________________________ >>> >> Wtr-development mailing list >>> >> [email protected] >>> >> http://rubyforge.org/mailman/listinfo/wtr-development >>> > >>> > >>> > -- >>> > Bret Pettichord >>> > Lead Developer, Watir, www.watir.com >>> > >>> > Blog, www.io.com/~wazmo/blog >>> > Twitter, www.twitter.com/bpettichord >>> > >>> > >>> > _______________________________________________ >>> > Wtr-development mailing list >>> > [email protected] >>> > http://rubyforge.org/mailman/listinfo/wtr-development >>> > >>> _______________________________________________ >>> Wtr-development mailing list >>> [email protected] >>> http://rubyforge.org/mailman/listinfo/wtr-development >> >> >> -- >> Bret Pettichord >> Lead Developer, Watir, www.watir.com >> >> Blog, www.io.com/~wazmo/blog >> Twitter, www.twitter.com/bpettichord >> > > > > -- > Bret Pettichord > Lead Developer, Watir, www.watir.com > > Blog, www.io.com/~wazmo/blog > Twitter, www.twitter.com/bpettichord > > > _______________________________________________ > Wtr-development mailing list > [email protected] > http://rubyforge.org/mailman/listinfo/wtr-development > _______________________________________________ Wtr-development mailing list [email protected] http://rubyforge.org/mailman/listinfo/wtr-development
