The current v3.1 builds are definitely stable enough to ditch v3.0.6. I am still putting them through the wringer and finding minor issues but we have been using v3.1 in production for a couple months.
Unfortunately the same issues will happen with v3.1 as with v3.0.6. It is also likely to be an issue with most any other .Net test framework that uses .Net remoting. I've sent you an email with a detailed explanation. The good news is that for v3.2 I am planning to drop .Net remoting and use a different communication infrastructure that will not have this issue anymore. Anyways, let's take this off-list for now. Jeff. -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Mark Kharitonov Sent: Monday, August 17, 2009 8:07 AM To: MbUnit.User Subject: MbUnit Re: A very strange behaviour when debugging an mbunit unit test with TD.NET Guys. I am really stuck here. Suddenly TD.NET tells me "Couldn't find test targets." And there is huge number of first chance exceptions when the tests run, 3 FileNotFoundException and 2 SerializationException per each test step. It will not be overstatement to say that it is a nightmare right now. Jeff, is 3.1 stable enough to ditch 3.06 altogether and use 3.1 instead? Thanks, On Aug 17, 2:11 am, "Jeff Brown" <[email protected]> wrote: > There are a couple of issues: > > 1. Gallio only registers specific versions of specific frameworks with > TDNet. For example, if you install Gallio v3.1, then it will tell > TDNet that it supports MbUnit v2.4.*, MbUnit v3.1.*, NUnit v2.4.8.*, > NUnit v2.5.*, etc... > > Consequently TDNet won't even call Gallio if someone asks to run a > test in an unsupported framework, like MbUnit v3.0.*. That's the > whole point of course. > > 2. Within Gallio, it decides which framework adapter to involve based > on the test file name (*.dll, *_spec.rb, etc.) and the version of any > referenced .Net assemblies (when applicable). So internally if you > feed Gallio v3.1 an MbUnit v3.0 test assembly then it will not find a > matching framework adapter so it will do nothing. > > I think the solution is to apply the following heuristic both in TDNet > and in Gallio. > > Heuristic: > > a. If a test file is supported by a registered framework use that framework. > > Otherwise, > > b. If a test file would be supported by a different version of a > registered framework, emit an error to the user describing the version > mismatch. Do not invoke the framework. > > Otherwise, > > c. Perform some default behavior. (In TDNet's case, use the AdHoc > runner, in Gallio's case, emit an error.) > > How does that sound? > > I will implement this heuristic in Gallio v3.1 but for best results it > also needs to be in TDNet unless I make Gallio omit framework version > constraints from its TDNet registration (not preferred). > > Jeff. > > -----Original Message----- > From: [email protected] [mailto:[email protected]] > On > > Behalf Of Jamie Cansdale > Sent: Saturday, August 15, 2009 4:32 AM > To: [email protected] > Subject: MbUnit Re: A very strange behaviour when debugging an mbunit > unit test with TD.NET > > > Could we just not run the ad-hoc test runner if some other runner > > claims the test? > > If some other runner executes or flags as ignored a single test then > the ad-hoc *shouldn't* run. It looks a bit like the test events aren't > being reported back to TestDriven.Net. > > > installation issue or versioning conflict and TDNet's ad-hoc runner > > is taking over the scene. > > What we really need is a way to warn the user when an incompatible > version of MbUnit is being targeted. If after failing to find any > tests Gallio could check the target method for any known attributes > (of the wrong version). It could then flag the test as ignored and > give the user a helpful message saying where to download the correct > version. I could add similar logic to the ad hoc runner in case the user was using an older version of Gallio. > > Does that sound like plan? > > Regards, > Jamie. > > --http://www.testdriven.nethttp://twitter.com/jcansdalehttp://weblogs. > asp.net/nunitaddin > > On Sat, Aug 15, 2009 at 11:50 AM, Jeff Brown<[email protected]> wrote: > > > Could we just not run the ad-hoc test runner if some other runner > > claims the test? > > > I tried hunting around for the code that defaults to the ad-hoc > > runner but did not find it. It seems the behavior is different > > across TDNet > versions. > > > I can't really overemphasise just how much of a nuisance the ad-hoc > > runner is for me. People keep complaining that MbUnit is broken > > because data-driven tests aren't running when in fact there's some > > other installation issue or versioning conflict and TDNet's ad-hoc > > runner is taking over the scene. > > > Jeff. > > > -----Original Message----- > > From: [email protected] > > [mailto:[email protected]] > > On Behalf Of Jamie Cansdale > > Sent: Saturday, August 15, 2009 2:24 AM > > To: [email protected] > > Subject: MbUnit Re: A very strange behaviour when debugging an > > mbunit unit test with TD.NET > > > Hi Mark, > > > It looks like TestDriven.Net isn't being informed about the Gallio > > test that ran. Because it thinks no tests have been executed yet, it > > goes on the execute the method as an 'ad hoc' test (seehttp://bit.ly/rbMDc). > > > Could you try copying you test project and stripping it down until > > you have a minimal repro? > > > Regards, > > Jamie. > > > -- > >http://www.testdriven.net > >http://twitter.com/jcansdale > >http://weblogs.asp.net/nunitaddin > > > On Sat, Aug 15, 2009 at 11:11 AM, Mark > > Kharitonov<[email protected]> wrote: > > >> I am attaching VS snapshot - > >>http://groups.google.com/group/MbUnitUser/web/VSSnapShot.JPG > > >> The snapshot displays 4 areas: > >> * The source code editor, stopped on a breakpoint. Note the ctx > >> parameter to the test method. > >> * The watch window, displaying the ctx parameter. Note the value is > >> null. > >> * The test output pane. Note, that the test has already finished > >> and the test report is ready! > >> * The Gallio test report, where one can clearly see that the test > >> has already finished and succeeded. > > >> HOWEVER, TD.NET runs the test method one more time passing null in > >> the ctx parameter. This is done AFTER the test is finished from > >> Gallio point of view - it has already created a test report > >> indicating the success of the test. > >> Needless to say, that the test factory method - > >> EnumerateAllHierarchyPoliciesWithRoot never yields the null reference. > >> This is extremely strange and I could not reproduce it with a > >> trivial example. > > >> The complete test pane output looks like this: > >> =================================================================== > >> == > >> = > >> = > >> ------ Test started: Assembly: Shunra.Infra.Test.dll ------ > > >> Gallio TestDriven.Net Runner - Version 3.0.6 build 787 > > >> Test Assemblies: > >> C:\Dev\windows\Infra\Shunra.Infra.Test\..\..\bin\Debug > >> \Shunra.Infra.Test.dll > > >> Start time: 11:43 AM > >> Verifying assembly names. > >> Initializing the test runner. > >> Running the tests. > >> [warning] The test was ignored. > >> Location: > >> C:\Dev\windows\Infra\Shunra.Infra.Test\EntitiesAssemblies.cs > >> (122) > >> Disposing the test runner. > >> Stop time: 11:46 AM (Total execution time: 188.520 seconds) > > >> Test Report: > >> file:///C:/Documents%20and%20Settings/mark.kharitonov/Local%20Setti > >> ng s /Temp/Gallio.TDNetRunner/Shunra.Infra.Test.dll.html > >> =================================================================== > >> == > >> = > >> = > > >> Note, that no test steps are displayed, despite the fact that 7 > >> test steps were executed - they are clearly reported in the Gallio > >> Test Report. > > >> Another strange thing is the call stack at this moment (and I > >> remind you, that the test is already finished as far as Gallio is > >> concerned) Here it is: > >> =================================================================== > >> == > >> = > >> = > > >>> Shunra.Infra.Test.dll!Shunra.Infra.Test.Entities.RootIsDefined(Shu > >>> nr a .Infra.Test.IEntityContext ctx = null) Line 436 C# > >> [Native to Managed Transition] > >> [Managed to Native Transition] > >> TestDriven.AdHoc.dll! > >> TestDriven.AdHoc.TestRunner.AdHocTestRunner.runAdHoc > >> (TestDriven.TestRunner.Framework.ITestListener testListener = > >> {TestDriven.TestRunner.ThreadTestRunner.ThreadTestListener}, > >> TestDriven.TestRunner.Framework.ITraceListener traceListener = > >> {TestDriven.TestRunner.ThreadTestRunner.ThreadTraceListener}, > >> string assemblyPath = > >> "C:\\Dev\\windows\\Infra\\Shunra.Infra.Test\\..\\..\\bin > >> \\Debug\\Shunra.Infra.Test.dll", string cref = > >> "M:Shunra.Infra.Test.Entities.RootIsDefined > >> (Shunra.Infra.Test.IEntityContext)") + 0x45a bytes > > >> TestDriven.AdHoc.dll!TestDriven.AdHoc.TestRunner.AdHocTestRunner.Ru > >> n (TestDriven.TestRunner.Framework.ITestListener testListener = > >> {TestDriven.TestRunner.ThreadTestRunner.ThreadTestListener}, > >> TestDriven.TestRunner.Framework.ITraceListener traceListener = > >> {TestDriven.TestRunner.ThreadTestRunner.ThreadTraceListener}, > >> string assemblyPath = > >> "C:\\Dev\\windows\\Infra\\Shunra.Infra.Test\\..\\..\\bin > >> \\Debug\\Shunra.Infra.Test.dll", string testPath = > >> "M:Shunra.Infra.Test.Entities.RootIsDefined > >> (Shunra.Infra.Test.IEntityContext)") + 0x8c bytes > > >> TestDriven.TestRunner.dll!TestDriven.TestRunner.AdaptorTestRunner.R > >> un (TestDriven.TestRunner.Framework.ITestListener testListener = > >> {TestDriven.TestRunner.ThreadTestRunner.ThreadTestListener}, > >> TestDriven.TestRunner.Framework.ITraceListener traceListener = > >> {TestDriven.TestRunner.ThreadTestRunner.ThreadTraceListener}, > >> string assemblyPath = > >> "C:\\Dev\\windows\\Infra\\Shunra.Infra.Test\\..\\..\\bin > >> \\Debug\\Shunra.Infra.Test.dll", string testPath = > >> "M:Shunra.Infra.Test.Entities.RootIsDefined > >> (Shunra.Infra.Test.IEntityContext)") + 0xb8 bytes > >> TestDriven.TestRunner.dll! > >> TestDriven.TestRunner.ThreadTestRunner.Runner.Run() + 0x68 bytes > >> > >> mscorlib.dll!System.Threading.ThreadHelper.ThreadStart_Context > >> (object state = {System.Threading.ThreadHelper}) + 0xac bytes > >> mscorlib.dll!System.Threading.ExecutionContext.Run > >> (System.Threading.ExecutionContext executionContext, > >> System.Threading.ContextCallback callback, object state) + 0x5a > >> bytes > >> mscorlib.dll!System.Threading.ThreadHelper.ThreadStart() + > >> 0x61 bytes > >> [Appdomain Transition] > >> =================================================================== > >> == > >> = > >> = > > >> Unfortunately, I am unable to reproduce it in a trivial example, > >> but in my solution it is 100% reproducable. > >> Any ideas? > >> Thanks. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "MbUnit.User" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/mbunituser?hl=en -~----------~----~----~----~------~----~------~--~---
