This is so weird. Any chance you could ship me a bundle of code with your extension?
I'm sorry I haven't been too helpful thus far. Jeff. -----Original Message----- From: [email protected] [mailto:[EMAIL PROTECTED] On Behalf Of Aaron Rehaag Sent: Friday, February 08, 2008 8:16 AM To: MbUnit.User Subject: MbUnit Re: Assertions going unhandled by MbUnit when using parameterized run invokers I've put breakpoints on the lines before and after, as well as on the Assert.Fail() in the above code. The before breakpoint triggers, the Assert.Fail() breakpoint line triggers, the after ones don't. I agree that my code isn't particularly robust (I was planning a cleanup pass once I got it working), but as far as I can tell the cppunit process is running correctly, and terminating with the correct results. Here's the exeception output captured via the GUI's output: MbUnit.Core.Exceptions.AssertionException ----- Message: CppUnit failure! In f:\interop\cppunit\suite1.cpp:8 assertion failed - Expression: false Type: MbUnit.Core.Exceptions.AssertionException Source: MbUnit.Framework TargetSite: Void Fail(System.String) HelpLink: null Stack: at MbUnit.Framework.Assert.Fail(String message) at MbUnit.AddIn.Interop.CppUnit.CppUnitRunInvoker.Execute(Object o, IList args) in F:\Interop\mbUnit\CppUnit\CppUnitRunInvoker.cs:line 81 at MbUnit.Core.RunPipeStarter.Run(Object fixture, Boolean IsExplicit) And here it is again from the dialog prompted by the AutoRunner... AssertionException was unhandled by user code ----- CppUnit failure! In f:\interop\cppunit\suite1.cpp:8 assertion failed - Expression: false Here's the code for the CppUnitRun: public class CppUnitRun : IRun { #region IRun members public bool IsTest { get { return false; } } public string Name { get { return "CppUnitRun"; } } public void Reflect(RunInvokerTree tree, RunInvokerVertex parent, Type subjectType) { try { CppUnitFixtureAttribute fixture = (CppUnitFixtureAttribute)(subjectType.GetCustomAttributes(true)[0]); foreach (TestDetails test in GetTestList(fixture.Path)) { IRunInvoker testInvoker = new CppUnitRunInvoker(this, fixture.Path, test.Suite, test.Test); tree.AddChild(parent, testInvoker); } } catch { } } #endregion private IEnumerable GetTestList(string path) { IList testList = new ArrayList(); Process process = new Process(); process.StartInfo.FileName = path; process.StartInfo.Arguments = "-list"; process.StartInfo.UseShellExecute = false; process.StartInfo.RedirectStandardOutput = true; process.Start(); string testListXml = process.StandardOutput.ReadToEnd(); process.WaitForExit(); XmlDocument xmlDoc = new XmlDocument(); xmlDoc.LoadXml(testListXml); foreach (XmlNode node in xmlDoc.FirstChild) { string testName = node.InnerText; testName = testName.Replace("::", ":"); string[] testDetails = testName.Split(':'); testList.Add(new TestDetails(testDetails[0], testDetails[1])); } return testList; } private struct TestDetails { public string Suite; public string Test; public TestDetails(string suite, string test) { Suite = suite; Test = test; } }; } Unless you can spot something particularly weird in the code above, I think I'm going to have to bite the bullet and download the MbUnit source and step through it in a debugger. The results I'm seeing just aren't making sense to me given the code in RunPipeStarter, and I'd kinda like to see why that catch(Exception) block isn't detecting this case, given the AssertionException seems to derive from Exception. Cheers, -A On Feb 7, 10:48 pm, "Jeff Brown" <[EMAIL PROTECTED]> wrote: > Thanks for the detailed description. > > Have you verified that your CppUnitRunInvoker is really calling > Assert.Fail when it runs inside AutoRunner or the GUI? You could do > this by launching the tests and attaching a debugger. Or you could > log some diagnostic output to check. > > I say this because I notice you don't check the exit code of the > CppUnit test runner process. As long as it emits some XML, the code > assumes all went well unless a <FailedTest> element is found. It's > possible that CppUnit failed to run the test for some other reason. > > For example, I also notice you don't set the WorkingDirectory of the > process. Alternately because the StandardError stream is not being > redirected or consumed, it may interact with the calling command shell. > Weird things could happen. > > Alternately, it could be that you CppUnitRunInvoker is being decorated > by some other run invoker that is eating the exception. I can't tell > whether that's the case because I don't have the code for the > CppUnitRun. I assumes it's unlikely. > > The only other thing I can think of is a ThreadAbortedException > wreaking havoc. But you would probably see that in the test output just the > same. > > What does MbUnit actually display in the test report? Does it just > look like everything passed or are you observing some other kind of > failure? Do the tests even appear in the report? So far you've only > said that it doesn't look like the AssertionException gets caught. > What actually happens? > > Jeff. > > > > -----Original Message----- > From: [email protected] [mailto:[EMAIL PROTECTED] > On > > Behalf Of Aaron Rehaag > Sent: Thursday, February 07, 2008 8:57 PM > To: MbUnit.User > Subject: MbUnit Re: Assertions going unhandled by MbUnit when using > parameterized run invokers > > Jeff, > > Long term, I'd be happy to port my solution to Gallio and contribute > it to the MbUnit project. I'd be happy to discuss it with you more > offline. In the meantime, I do need to find a stop-gap solution so I > can use the technology at my workplace. > > Based off the code I see in RunPipeStarter, it sure seems that the > AssertionFaliureException should be getting caught,but my tests > definitely show otherwise. As requested, I'll provide more details. > > The basic idea of my solution is fairly straightforward. There are 2 > major > steps: > > 1) Mirror the cppunit test hierarchy in MbUnit using a custom IRun > (CppUnitRun). Basically, we launch the cppunit test runner executable > passing a "-list" command line option. When ran in this fashion, the > test hierarchy is spit out in a custom XML format to cout, which > CppUnitRun captures, and parses, adding tests to duplicate the > hierarchy. This appears to be working correctly, so I'll spare you the > details of this code. > > 2) We execute the tests using a custom IRunInvoker > (CppUnitRunInvoker). The invoker launches the test runner executable > once per test, passing the test's name as a command line option. In > response, the executable spits out the test results (using CppUnit's > built-in XMLOutputter) to cout, which are captured by the invoker and > parsed. If the results indicate the test has failed, an Assert.Fail() > is raised, with a message indicating details indicating why the test failed. > Here's the code: > > public class CppUnitRunInvoker : IRunInvoker > { > public CppUnitRunInvoker(IRun generator, string path, string > suite, string test) > { > _generator = generator; > _suite = suite; > _test = test; > _path = path; > } > > public IRun Generator > { > get { return _generator; } > } > > public string Name > { > get { return (_suite + "." + _test); } > } > > public bool ContainsMemberInfo(MemberInfo memberInfo) > { > foreach (MemberInfo member in GetType().GetMembers()) > { > if (member.Equals(memberInfo)) > return true; > } > > return false; > } > > public object Execute(object o, IList args) > { > Process process = new Process(); > > process.StartInfo.FileName = _path; > process.StartInfo.Arguments = "-run " + _suite + "::" + > _test; > process.StartInfo.UseShellExecute = false; > process.StartInfo.RedirectStandardOutput = true; > process.Start(); > > string testRunXml = process.StandardOutput.ReadToEnd(); > testRunXml = TrimStartingWhitespace(testRunXml); > process.WaitForExit(); > > XmlDocument xmlDoc = new XmlDocument(); > xmlDoc.LoadXml(testRunXml); > XmlNode failedTests = > xmlDoc.FirstChild.NextSibling.FirstChild; > if (failedTests.HasChildNodes) // If the test failed. > { > string file = ""; > string line = ""; > string message = ""; > > foreach (XmlNode node in > failedTests.FirstChild.ChildNodes) > { > if (node.Name == "Location") > { > file = node.FirstChild.InnerText; > line = node.FirstChild.NextSibling.InnerText; > } > else if (node.Name == "Message") > { > message = node.InnerText; > } > } > > MbUnit.Framework.Assert.Fail("\n\nCppUnit failure!\n \nIn " > + file + ":" + line + "\n\n" + message); // --- BOOM! --- > } > > return 0; > } > > private string TrimStartingWhitespace(string str) > { > return str.TrimStart(new char[]{'\r','\n','\t',' '}); > } > > private IRun _generator; > private string _path; > private string _suite; > private string _test; > } > > Other than that, there's not much to it. Here's the test fixture > itself: > > [CppUnitFixture("F:\\Interop\\cppunit\\debug\\unit.exe")] > public class CppUnitSuite1 > { > } > > The implementation of the CppUnitFixtureAttribute is fairly > straightforward, so I'm not going to duplicate it here. I'll also > spare you the joys of the test runner c++ code, but you might want the > following sample test runner output to make sense of the XML parsing > above: > > <?xml version="1.0" encoding='ISO-8859-1' standalone='yes' ?> > <TestRun> > <FailedTests> > <FailedTest id="1"> > <Name>Suite1::Test1</Name> > <FailureType>Assertion</FailureType> > <Location> > <File>f:\interop\cppunit\suite1.cpp</File> > <Line>8</Line> > </Location> > <Message>assertion failed > - Expression: false > </Message> > </FailedTest> > </FailedTests> > <SuccessfulTests></SuccessfulTests> > <Statistics> > <Tests>1</Tests> > <FailuresTotal>1</FailuresTotal> > <Errors>0</Errors> > <Failures>1</Failures> > </Statistics> > </TestRun> > > As mentioned earlier, the Assert.Fail() raises the exception, which > seems to go unhandled when ran through the AutoRunner, and appears to > be getting caught by a catch block somewhere in the GUI when ran in that mode. > > Thoughts? > -A > > On Feb 4, 3:39 pm, Jeff Brown <[EMAIL PROTECTED]> wrote: > > Hrrm. > > Let's see. > > > 1. So, I can see in RunPipeStarter that all exceptions are caught > > when executing RunInvokers. There is no relation to arguments at > > all as far as I can tell. > > > I'm not also clear where the assertion is thrown since your sample > > below does not show one. Is it inside the test method? Does your > > test method actually have arguments? Changing whether arguments are > > added will effect whether the test method can actually be invoked. > > > I think it would help me to see more code so I can get a more > > complete idea of what's going on. > > > 2. Would you be interested in porting this work to Gallio? I think > > you'll find it much more extensible and expressive. It also runs > > MbUnit v2, NUnit and xUnit.Net tests via an adapter. I'm currently > > working to get the Alpha 2 release out the door. > > > Once that's done we can look into adding fully supported CppUnit, > > CSUnit, MSTest adapters among others. They would probably only take > > 1 day each depending on the level of integration desired. > > > Jeff. > > > -----Original Message----- > > From: [email protected] > > [mailto:[EMAIL PROTECTED] > > > On Behalf Of Aaron Rehaag > > Sent: Sunday, February 03, 2008 12:07 AM > > To: MbUnit.User > > Subject: MbUnit Assertions going unhandled by MbUnit when using > > parameterized run invokers > > > Hey everyone, > > > I'm trying to build a custom extension to MbUnit to execute CppUnit > > tests and mirror the results into MbUnit test hierarchy. In order > > to do so, I've created a custom run invoker to execute tests using a > > cppunit runner. > > > Now the interesting part... when I add an Assert.Fail() statement > > into my run invoker' Execute() method, the AssertionException thrown > > is never caught / handled by MbUnit. As I suspected my code might > > be flawed, I tried downloading Jay Flower's XHtml extension sample > > solution > > (http://jayflowers.com/WordPress/?p=88) and found the same behavior > > running the 2.4 release. In a weird twist, it seems that if you > > comment out the following line: > > > public override object Execute(object o, IList args) > > { > > foreach (Object Arg in this.Args) > > { > > // args.Add(Arg); /* Including an argument somehow > > makes MbUnit not handle the assertion exception. */ > > } > > > return base.Execute(o, args); > > } > > > The assertion is caught by MbUnit. > > > Does anyone have any insight into why MbUnit is exhibiting this > > behavior? Am I missing some crucial logic for handling these cases > > correctly? > > > Any help that can be provided would be much appreciated. > > > Cheers, > > -A- Hide quoted text - > > > - Show quoted text -- Hide quoted text - > > - Show quoted text - --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "MbUnit.User" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/MbUnitUser?hl=en -~----------~----~----~----~------~----~------~--~---
