Thanks for the detailed description. Have you verified that your CppUnitRunInvoker is really calling Assert.Fail when it runs inside AutoRunner or the GUI? You could do this by launching the tests and attaching a debugger. Or you could log some diagnostic output to check.
I say this because I notice you don't check the exit code of the CppUnit test runner process. As long as it emits some XML, the code assumes all went well unless a <FailedTest> element is found. It's possible that CppUnit failed to run the test for some other reason. For example, I also notice you don't set the WorkingDirectory of the process. Alternately because the StandardError stream is not being redirected or consumed, it may interact with the calling command shell. Weird things could happen. Alternately, it could be that you CppUnitRunInvoker is being decorated by some other run invoker that is eating the exception. I can't tell whether that's the case because I don't have the code for the CppUnitRun. I assumes it's unlikely. The only other thing I can think of is a ThreadAbortedException wreaking havoc. But you would probably see that in the test output just the same. What does MbUnit actually display in the test report? Does it just look like everything passed or are you observing some other kind of failure? Do the tests even appear in the report? So far you've only said that it doesn't look like the AssertionException gets caught. What actually happens? Jeff. -----Original Message----- From: [email protected] [mailto:[EMAIL PROTECTED] On Behalf Of Aaron Rehaag Sent: Thursday, February 07, 2008 8:57 PM To: MbUnit.User Subject: MbUnit Re: Assertions going unhandled by MbUnit when using parameterized run invokers Jeff, Long term, I'd be happy to port my solution to Gallio and contribute it to the MbUnit project. I'd be happy to discuss it with you more offline. In the meantime, I do need to find a stop-gap solution so I can use the technology at my workplace. Based off the code I see in RunPipeStarter, it sure seems that the AssertionFaliureException should be getting caught,but my tests definitely show otherwise. As requested, I'll provide more details. The basic idea of my solution is fairly straightforward. There are 2 major steps: 1) Mirror the cppunit test hierarchy in MbUnit using a custom IRun (CppUnitRun). Basically, we launch the cppunit test runner executable passing a "-list" command line option. When ran in this fashion, the test hierarchy is spit out in a custom XML format to cout, which CppUnitRun captures, and parses, adding tests to duplicate the hierarchy. This appears to be working correctly, so I'll spare you the details of this code. 2) We execute the tests using a custom IRunInvoker (CppUnitRunInvoker). The invoker launches the test runner executable once per test, passing the test's name as a command line option. In response, the executable spits out the test results (using CppUnit's built-in XMLOutputter) to cout, which are captured by the invoker and parsed. If the results indicate the test has failed, an Assert.Fail() is raised, with a message indicating details indicating why the test failed. Here's the code: public class CppUnitRunInvoker : IRunInvoker { public CppUnitRunInvoker(IRun generator, string path, string suite, string test) { _generator = generator; _suite = suite; _test = test; _path = path; } public IRun Generator { get { return _generator; } } public string Name { get { return (_suite + "." + _test); } } public bool ContainsMemberInfo(MemberInfo memberInfo) { foreach (MemberInfo member in GetType().GetMembers()) { if (member.Equals(memberInfo)) return true; } return false; } public object Execute(object o, IList args) { Process process = new Process(); process.StartInfo.FileName = _path; process.StartInfo.Arguments = "-run " + _suite + "::" + _test; process.StartInfo.UseShellExecute = false; process.StartInfo.RedirectStandardOutput = true; process.Start(); string testRunXml = process.StandardOutput.ReadToEnd(); testRunXml = TrimStartingWhitespace(testRunXml); process.WaitForExit(); XmlDocument xmlDoc = new XmlDocument(); xmlDoc.LoadXml(testRunXml); XmlNode failedTests = xmlDoc.FirstChild.NextSibling.FirstChild; if (failedTests.HasChildNodes) // If the test failed. { string file = ""; string line = ""; string message = ""; foreach (XmlNode node in failedTests.FirstChild.ChildNodes) { if (node.Name == "Location") { file = node.FirstChild.InnerText; line = node.FirstChild.NextSibling.InnerText; } else if (node.Name == "Message") { message = node.InnerText; } } MbUnit.Framework.Assert.Fail("\n\nCppUnit failure!\n \nIn " + file + ":" + line + "\n\n" + message); // --- BOOM! --- } return 0; } private string TrimStartingWhitespace(string str) { return str.TrimStart(new char[]{'\r','\n','\t',' '}); } private IRun _generator; private string _path; private string _suite; private string _test; } Other than that, there's not much to it. Here's the test fixture itself: [CppUnitFixture("F:\\Interop\\cppunit\\debug\\unit.exe")] public class CppUnitSuite1 { } The implementation of the CppUnitFixtureAttribute is fairly straightforward, so I'm not going to duplicate it here. I'll also spare you the joys of the test runner c++ code, but you might want the following sample test runner output to make sense of the XML parsing above: <?xml version="1.0" encoding='ISO-8859-1' standalone='yes' ?> <TestRun> <FailedTests> <FailedTest id="1"> <Name>Suite1::Test1</Name> <FailureType>Assertion</FailureType> <Location> <File>f:\interop\cppunit\suite1.cpp</File> <Line>8</Line> </Location> <Message>assertion failed - Expression: false </Message> </FailedTest> </FailedTests> <SuccessfulTests></SuccessfulTests> <Statistics> <Tests>1</Tests> <FailuresTotal>1</FailuresTotal> <Errors>0</Errors> <Failures>1</Failures> </Statistics> </TestRun> As mentioned earlier, the Assert.Fail() raises the exception, which seems to go unhandled when ran through the AutoRunner, and appears to be getting caught by a catch block somewhere in the GUI when ran in that mode. Thoughts? -A On Feb 4, 3:39 pm, Jeff Brown <[EMAIL PROTECTED]> wrote: > Hrrm. > Let's see. > > 1. So, I can see in RunPipeStarter that all exceptions are caught when > executing RunInvokers. There is no relation to arguments at all as > far as I can tell. > > I'm not also clear where the assertion is thrown since your sample > below does not show one. Is it inside the test method? Does your > test method actually have arguments? Changing whether arguments are > added will effect whether the test method can actually be invoked. > > I think it would help me to see more code so I can get a more complete > idea of what's going on. > > 2. Would you be interested in porting this work to Gallio? I think > you'll find it much more extensible and expressive. It also runs > MbUnit v2, NUnit and xUnit.Net tests via an adapter. I'm currently > working to get the Alpha 2 release out the door. > > Once that's done we can look into adding fully supported CppUnit, > CSUnit, MSTest adapters among others. They would probably only take 1 > day each depending on the level of integration desired. > > Jeff. > > > > -----Original Message----- > From: [email protected] [mailto:[EMAIL PROTECTED] > > On Behalf Of Aaron Rehaag > Sent: Sunday, February 03, 2008 12:07 AM > To: MbUnit.User > Subject: MbUnit Assertions going unhandled by MbUnit when using > parameterized run invokers > > Hey everyone, > > I'm trying to build a custom extension to MbUnit to execute CppUnit > tests and mirror the results into MbUnit test hierarchy. In order to > do so, I've created a custom run invoker to execute tests using a > cppunit runner. > > Now the interesting part... when I add an Assert.Fail() statement into > my run invoker' Execute() method, the AssertionException thrown is > never caught / handled by MbUnit. As I suspected my code might be > flawed, I tried downloading Jay Flower's XHtml extension sample > solution > (http://jayflowers.com/WordPress/?p=88) and found the same behavior > running the 2.4 release. In a weird twist, it seems that if you > comment out the following line: > > public override object Execute(object o, IList args) > { > foreach (Object Arg in this.Args) > { > // args.Add(Arg); /* Including an argument somehow makes > MbUnit not handle the assertion exception. */ > } > > return base.Execute(o, args); > } > > The assertion is caught by MbUnit. > > Does anyone have any insight into why MbUnit is exhibiting this > behavior? Am I missing some crucial logic for handling these cases > correctly? > > Any help that can be provided would be much appreciated. > > Cheers, > -A- Hide quoted text - > > - Show quoted text - --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "MbUnit.User" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/MbUnitUser?hl=en -~----------~----~----~----~------~----~------~--~---
