There are a couple of issues: 1. Gallio only registers specific versions of specific frameworks with TDNet. For example, if you install Gallio v3.1, then it will tell TDNet that it supports MbUnit v2.4.*, MbUnit v3.1.*, NUnit v2.4.8.*, NUnit v2.5.*, etc...
Consequently TDNet won't even call Gallio if someone asks to run a test in an unsupported framework, like MbUnit v3.0.*. That's the whole point of course. 2. Within Gallio, it decides which framework adapter to involve based on the test file name (*.dll, *_spec.rb, etc.) and the version of any referenced .Net assemblies (when applicable). So internally if you feed Gallio v3.1 an MbUnit v3.0 test assembly then it will not find a matching framework adapter so it will do nothing. I think the solution is to apply the following heuristic both in TDNet and in Gallio. Heuristic: a. If a test file is supported by a registered framework use that framework. Otherwise, b. If a test file would be supported by a different version of a registered framework, emit an error to the user describing the version mismatch. Do not invoke the framework. Otherwise, c. Perform some default behavior. (In TDNet's case, use the AdHoc runner, in Gallio's case, emit an error.) How does that sound? I will implement this heuristic in Gallio v3.1 but for best results it also needs to be in TDNet unless I make Gallio omit framework version constraints from its TDNet registration (not preferred). Jeff. -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Jamie Cansdale Sent: Saturday, August 15, 2009 4:32 AM To: [email protected] Subject: MbUnit Re: A very strange behaviour when debugging an mbunit unit test with TD.NET > Could we just not run the ad-hoc test runner if some other runner > claims the test? > If some other runner executes or flags as ignored a single test then the ad-hoc *shouldn't* run. It looks a bit like the test events aren't being reported back to TestDriven.Net. > installation issue or versioning conflict and TDNet's ad-hoc runner is > taking over the scene. > What we really need is a way to warn the user when an incompatible version of MbUnit is being targeted. If after failing to find any tests Gallio could check the target method for any known attributes (of the wrong version). It could then flag the test as ignored and give the user a helpful message saying where to download the correct version. I could add similar logic to the ad hoc runner in case the user was using an older version of Gallio. Does that sound like plan? Regards, Jamie. -- http://www.testdriven.net http://twitter.com/jcansdale http://weblogs.asp.net/nunitaddin On Sat, Aug 15, 2009 at 11:50 AM, Jeff Brown<[email protected]> wrote: > > Could we just not run the ad-hoc test runner if some other runner > claims the test? > > I tried hunting around for the code that defaults to the ad-hoc runner > but did not find it. It seems the behavior is different across TDNet versions. > > I can't really overemphasise just how much of a nuisance the ad-hoc > runner is for me. People keep complaining that MbUnit is broken > because data-driven tests aren't running when in fact there's some > other installation issue or versioning conflict and TDNet's ad-hoc > runner is taking over the scene. > > Jeff. > > -----Original Message----- > From: [email protected] [mailto:[email protected]] > On Behalf Of Jamie Cansdale > Sent: Saturday, August 15, 2009 2:24 AM > To: [email protected] > Subject: MbUnit Re: A very strange behaviour when debugging an mbunit > unit test with TD.NET > > > Hi Mark, > > It looks like TestDriven.Net isn't being informed about the Gallio > test that ran. Because it thinks no tests have been executed yet, it > goes on the execute the method as an 'ad hoc' test (see http://bit.ly/rbMDc). > > Could you try copying you test project and stripping it down until you > have a minimal repro? > > Regards, > Jamie. > > -- > http://www.testdriven.net > http://twitter.com/jcansdale > http://weblogs.asp.net/nunitaddin > > > > On Sat, Aug 15, 2009 at 11:11 AM, Mark > Kharitonov<[email protected]> wrote: >> >> I am attaching VS snapshot - >> http://groups.google.com/group/MbUnitUser/web/VSSnapShot.JPG >> >> The snapshot displays 4 areas: >> * The source code editor, stopped on a breakpoint. Note the ctx >> parameter to the test method. >> * The watch window, displaying the ctx parameter. Note the value is >> null. >> * The test output pane. Note, that the test has already finished and >> the test report is ready! >> * The Gallio test report, where one can clearly see that the test has >> already finished and succeeded. >> >> HOWEVER, TD.NET runs the test method one more time passing null in >> the ctx parameter. This is done AFTER the test is finished from >> Gallio point of view - it has already created a test report >> indicating the success of the test. >> Needless to say, that the test factory method - >> EnumerateAllHierarchyPoliciesWithRoot never yields the null reference. >> This is extremely strange and I could not reproduce it with a trivial >> example. >> >> The complete test pane output looks like this: >> ===================================================================== >> = >> = >> ------ Test started: Assembly: Shunra.Infra.Test.dll ------ >> >> Gallio TestDriven.Net Runner - Version 3.0.6 build 787 >> >> Test Assemblies: >> C:\Dev\windows\Infra\Shunra.Infra.Test\..\..\bin\Debug >> \Shunra.Infra.Test.dll >> >> Start time: 11:43 AM >> Verifying assembly names. >> Initializing the test runner. >> Running the tests. >> [warning] The test was ignored. >> Location: >> C:\Dev\windows\Infra\Shunra.Infra.Test\EntitiesAssemblies.cs >> (122) >> Disposing the test runner. >> Stop time: 11:46 AM (Total execution time: 188.520 seconds) >> >> Test Report: >> file:///C:/Documents%20and%20Settings/mark.kharitonov/Local%20Setting >> s /Temp/Gallio.TDNetRunner/Shunra.Infra.Test.dll.html >> ===================================================================== >> = >> = >> >> Note, that no test steps are displayed, despite the fact that 7 test >> steps were executed - they are clearly reported in the Gallio Test >> Report. >> >> Another strange thing is the call stack at this moment (and I remind >> you, that the test is already finished as far as Gallio is concerned) >> Here it is: >> ===================================================================== >> = >> = >> >>> >>> Shunra.Infra.Test.dll!Shunra.Infra.Test.Entities.RootIsDefined(Shunr >>> a .Infra.Test.IEntityContext ctx = null) Line 436 C# >> [Native to Managed Transition] >> [Managed to Native Transition] >> TestDriven.AdHoc.dll! >> TestDriven.AdHoc.TestRunner.AdHocTestRunner.runAdHoc >> (TestDriven.TestRunner.Framework.ITestListener testListener = >> {TestDriven.TestRunner.ThreadTestRunner.ThreadTestListener}, >> TestDriven.TestRunner.Framework.ITraceListener traceListener = >> {TestDriven.TestRunner.ThreadTestRunner.ThreadTraceListener}, string >> assemblyPath = >> "C:\\Dev\\windows\\Infra\\Shunra.Infra.Test\\..\\..\\bin >> \\Debug\\Shunra.Infra.Test.dll", string cref = >> "M:Shunra.Infra.Test.Entities.RootIsDefined >> (Shunra.Infra.Test.IEntityContext)") + 0x45a bytes >> >> TestDriven.AdHoc.dll!TestDriven.AdHoc.TestRunner.AdHocTestRunner.Run >> (TestDriven.TestRunner.Framework.ITestListener testListener = >> {TestDriven.TestRunner.ThreadTestRunner.ThreadTestListener}, >> TestDriven.TestRunner.Framework.ITraceListener traceListener = >> {TestDriven.TestRunner.ThreadTestRunner.ThreadTraceListener}, string >> assemblyPath = >> "C:\\Dev\\windows\\Infra\\Shunra.Infra.Test\\..\\..\\bin >> \\Debug\\Shunra.Infra.Test.dll", string testPath = >> "M:Shunra.Infra.Test.Entities.RootIsDefined >> (Shunra.Infra.Test.IEntityContext)") + 0x8c bytes >> >> TestDriven.TestRunner.dll!TestDriven.TestRunner.AdaptorTestRunner.Run >> (TestDriven.TestRunner.Framework.ITestListener testListener = >> {TestDriven.TestRunner.ThreadTestRunner.ThreadTestListener}, >> TestDriven.TestRunner.Framework.ITraceListener traceListener = >> {TestDriven.TestRunner.ThreadTestRunner.ThreadTraceListener}, string >> assemblyPath = >> "C:\\Dev\\windows\\Infra\\Shunra.Infra.Test\\..\\..\\bin >> \\Debug\\Shunra.Infra.Test.dll", string testPath = >> "M:Shunra.Infra.Test.Entities.RootIsDefined >> (Shunra.Infra.Test.IEntityContext)") + 0xb8 bytes >> TestDriven.TestRunner.dll! >> TestDriven.TestRunner.ThreadTestRunner.Runner.Run() + 0x68 bytes >> mscorlib.dll!System.Threading.ThreadHelper.ThreadStart_Context >> (object state = {System.Threading.ThreadHelper}) + 0xac bytes >> mscorlib.dll!System.Threading.ExecutionContext.Run >> (System.Threading.ExecutionContext executionContext, >> System.Threading.ContextCallback callback, object state) + 0x5a bytes >> mscorlib.dll!System.Threading.ThreadHelper.ThreadStart() + >> 0x61 bytes >> [Appdomain Transition] >> ===================================================================== >> = >> = >> >> Unfortunately, I am unable to reproduce it in a trivial example, but >> in my solution it is 100% reproducable. >> Any ideas? >> Thanks. >> > >> > > > > > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "MbUnit.User" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/MbUnitUser?hl=en -~----------~----~----~----~------~----~------~--~---
