Sean and I talked about it for about 2 seconds in the Bar at cfO, for OpenBD. The consideration was an augmented engine that would record stats on statements executed then a small tool the analyze the output. Essentially everything is in ColdFusion to do this, as somehow they are correlating a line of code back to the debugger in eclipse (this is the heart of code coverage really). I just have no desire to figure it out and honestly since it is a closed source system I would expect the vendor to provide this to me.
If code coverage is not an option how do you show confidence and measure improvement on your systems? I have a heard time with this today. Adam Haskell On Sat, May 17, 2008 at 4:12 PM, Marc Esher <[EMAIL PROTECTED]> wrote: > > I understand what you're saying Brian, but i respectfully disagree. > Take java coverage tools: they're completely independent from the unit > test framework (at least the 2 i've used are); Clover, for example, > supports both junit4 and testng. I think technically coverage is even > independent of testing in general. for example, you could instrument > your code and run it in your app normally, and it could report what > percentage of your code was actually executed. i.e. the work required > to actually analyze coverage (source or bytecode instrumentation) > happens outside of the mechanism running the code (test framework or > real app). > > I used to think that you could only do coverage for CF by getting at > the generated bytecode. i even posted that as my feature request when > greg wilson blogged asking for feature requests a while back (I think > it was greg wilson.... i remember you took up all the good answers! so > poor folk like me were stuck with lame things like "code coverage".). > But now i'm not so sure. I think that you might be able to do this > with source instrumentation. I guess a first step would be figuring > out how they do it for java coverage tools that use this approach. > > From what I understand, they essentially create a new version of > every source file, with each statement modified to enable reporting on > whether that statement was executed. And by statement, i do not mean > line. I mean that if you have if(x==5 || y==null), then that has to > be broken down such that both parts of that conditional must be hit in > order for it to be considered 100%. if you always hit that if > statement with x==5 but never y == null, then it's technically not > 100% covered. > > To be sure, this is a both a real need and also a real challenge. Pity > the poor guy who actually takes a stab at it, because for 3 months > he's going to get nothing but bitching from the people using it. I can > hear it now: "dude, i know you slaved nights and weekends on this > coverage tool, and i know you're 300 times smarter than me, but dude, > it's reporting 59% when i'm abso-f**king-lutely sure it's at least > 62%". The horror.... And it'd take about .5 seconds after this > project gets its first download before some jerkoff says "man, why's > it so slow?" > > This here is another of those products, like a good IDE, that I'd be > willing to spend money on if Adobe were to provide it. But man, I > gotta believe that something like this is waaaaay down on the list, > particularly when so much effort is spent on the RIA side of the > tracks and not on the grungy code nuts-and-bolts side, which is where > a coverage tool surely lives. > > best, > > marc > > On Sat, May 17, 2008 at 2:03 PM, Brian Kotek <[EMAIL PROTECTED]> wrote: > > Isn't something like this dependent on the actual unit testing framework > > being used? If so, Adobe really couldn't do anything about it, I'd think > it > > would be up to the unit test framework authors to incorporate something > like > > this. > > > > On Sat, May 17, 2008 at 1:59 PM, Adam Haskell <[EMAIL PROTECTED]> > wrote: > >> > >> What you are looking for is code coverage tools and analytics. > ColdFusion, > >> disappointingly, has none. Are you listening Adobe??? This is something > we > >> need. > >> > >> > >> Adam Haskell > >> > >> > >> On Fri, May 16, 2008 at 4:05 PM, Ronan Lucio <[EMAIL PROTECTED]> > wrote: > >>> > >>> Hi, > >>> > >>> I'm not sure if I can post this subject here. If not, don't worry. > >>> > >>> Is there some applications to check the percentage of test X percentage > >>> of code and things like that? > >>> > >>> Explaining better: > >>> Upon a time I attended a forum about Ruby on Rails. > >>> The guy presented a solution to cover 100% of tests. > >>> In his solution, every commit to the code was checked if there are > tests > >>> written to that code. > >>> The program only commited the code if there were 100% of tests. > >>> > >>> Is there anything like that for ColdFusion? > >>> > >>> Thanks, > >>> Ronan > >>> > >>> > >> > >> > >> > > > > > > > > > > > > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "CFCDev" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cfcdev?hl=en -~----------~----~----~----~------~----~------~--~---
