On Thu, Jul 30, 2009 at 10:53 AM, dan nessett<[email protected]> wrote: > > True. However, knowing the coverage of parserTests and knowing which code > isn't even being visited by it is the first step in understanding where the > holes are in testing. Code coverage is a primitive metric. But, it's a place > to start. > > --- On Thu, 7/30/09, Victor Vasiliev <[email protected]> wrote: > >> From: Victor Vasiliev <[email protected]> >> Subject: Re: [Wikitech-l] parserTests code coverage statistics >> To: "Wikimedia developers" <[email protected]> >> Date: Thursday, July 30, 2009, 1:28 AM >> dan nessett wrote: >> > I decided to investigate how well parserTests >> exercises the MW code. So, I threw together a couple of >> MacGyver tools that use xdebug's code coverage capability >> and analyzed the results. The results are very, very >> preliminary, but I thought I would get them out so others >> can look them over. In the next couple of days I hope to >> post more detailed results and the tools themselves on the >> Mediawiki wiki. (If someone could tell me the appropriate >> page to use that would be useful. Otherwise, I will just >> create a page in my own namespace). >> > >> > The statistics (again very preliminary) are: >> > >> > Number of files exercised: 141 Number of lines >> in those files: 85606 >> > Lines covered: 59489 Lines not covered: >> 26117 Percentage covered: 0.694916244188 >> > >> > So, parserTests is getting (at best) about 70% code >> coverage. This is better than I expected, but still it means >> parserTests does not test 26117 lines of code. What I mean >> by "at best" is xdebug just notes whether a line of code is >> visited. It doesn't do any logic analysis on which branches >> are taken. Furthermore, parserTests may not visit some files >> that are critical to the operation of the MW software. >> Obviously, xdebug can only gather statistics on visited >> files. >> > >> > I want to emphasize that there may be errors in these >> results due to bad assumptions on my part or bad coding. >> However, it is a place to start. >> > >> >> Well, they are *parser* tests, they are not intended to >> cover >> Special:Version or something else. >> >> --vvv >> >> _______________________________________________ >> Wikitech-l mailing list >> [email protected] >> https://lists.wikimedia.org/mailman/listinfo/wikitech-l >> > > > > > _______________________________________________ > Wikitech-l mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/wikitech-l >
For more generic unit tests, check out the stuff in /t/ and /tests/ Those could probably use improvement. -Chad _______________________________________________ Wikitech-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikitech-l
