Right. I have looked at both t/ and tests/ and agree that they could use some work. But when starting on a trip its best to walk in one direction to start. Otherwise you end up going around in circles.
--- On Thu, 7/30/09, Chad <[email protected]> wrote: > From: Chad <[email protected]> > Subject: Re: [Wikitech-l] parserTests code coverage statistics > To: "Wikimedia developers" <[email protected]> > Date: Thursday, July 30, 2009, 8:18 AM > On Thu, Jul 30, 2009 at 10:53 AM, dan > nessett<[email protected]> > wrote: > > > > True. However, knowing the coverage of parserTests and > knowing which code isn't even being visited by it is the > first step in understanding where the holes are in testing. > Code coverage is a primitive metric. But, it's a place to > start. > > > > --- On Thu, 7/30/09, Victor Vasiliev <[email protected]> > wrote: > > > >> From: Victor Vasiliev <[email protected]> > >> Subject: Re: [Wikitech-l] parserTests code > coverage statistics > >> To: "Wikimedia developers" <[email protected]> > >> Date: Thursday, July 30, 2009, 1:28 AM > >> dan nessett wrote: > >> > I decided to investigate how well > parserTests > >> exercises the MW code. So, I threw together a > couple of > >> MacGyver tools that use xdebug's code coverage > capability > >> and analyzed the results. The results are very, > very > >> preliminary, but I thought I would get them out so > others > >> can look them over. In the next couple of days I > hope to > >> post more detailed results and the tools > themselves on the > >> Mediawiki wiki. (If someone could tell me the > appropriate > >> page to use that would be useful. Otherwise, I > will just > >> create a page in my own namespace). > >> > > >> > The statistics (again very preliminary) are: > >> > > >> > Number of files exercised: 141 Number of > lines > >> in those files: 85606 > >> > Lines covered: 59489 Lines not covered: > >> 26117 Percentage covered: 0.694916244188 > >> > > >> > So, parserTests is getting (at best) about > 70% code > >> coverage. This is better than I expected, but > still it means > >> parserTests does not test 26117 lines of code. > What I mean > >> by "at best" is xdebug just notes whether a line > of code is > >> visited. It doesn't do any logic analysis on which > branches > >> are taken. Furthermore, parserTests may not visit > some files > >> that are critical to the operation of the MW > software. > >> Obviously, xdebug can only gather statistics on > visited > >> files. > >> > > >> > I want to emphasize that there may be errors > in these > >> results due to bad assumptions on my part or bad > coding. > >> However, it is a place to start. > >> > > >> > >> Well, they are *parser* tests, they are not > intended to > >> cover > >> Special:Version or something else. > >> > >> --vvv > >> > >> _______________________________________________ > >> Wikitech-l mailing list > >> [email protected] > >> https://lists.wikimedia.org/mailman/listinfo/wikitech-l > >> > > > > > > > > > > _______________________________________________ > > Wikitech-l mailing list > > [email protected] > > https://lists.wikimedia.org/mailman/listinfo/wikitech-l > > > > For more generic unit tests, check out the stuff in /t/ and > /tests/ > Those could probably use improvement. > > -Chad > > _______________________________________________ > Wikitech-l mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/wikitech-l _______________________________________________ Wikitech-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikitech-l
