I've put breakpoints on the lines before and after, as well as on the
Assert.Fail() in the above code. The before breakpoint triggers, the
Assert.Fail() breakpoint line triggers, the after ones don't. I agree
that my code isn't particularly robust (I was planning a cleanup pass
once I got it working), but as far as I can tell the cppunit process
is running correctly, and terminating with the correct results.
Here's the exeception output captured via the GUI's output:
MbUnit.Core.Exceptions.AssertionException
-----
Message:
CppUnit failure!
In f:\interop\cppunit\suite1.cpp:8
assertion failed
- Expression: false
Type: MbUnit.Core.Exceptions.AssertionException
Source: MbUnit.Framework
TargetSite: Void Fail(System.String)
HelpLink: null
Stack: at MbUnit.Framework.Assert.Fail(String message)
at MbUnit.AddIn.Interop.CppUnit.CppUnitRunInvoker.Execute(Object
o, IList args) in F:\Interop\mbUnit\CppUnit\CppUnitRunInvoker.cs:line
81
at MbUnit.Core.RunPipeStarter.Run(Object fixture, Boolean
IsExplicit)
And here it is again from the dialog prompted by the AutoRunner...
AssertionException was unhandled by user code
-----
CppUnit failure!
In f:\interop\cppunit\suite1.cpp:8
assertion failed
- Expression: false
Here's the code for the CppUnitRun:
public class CppUnitRun : IRun
{
#region IRun members
public bool IsTest
{
get { return false; }
}
public string Name
{
get { return "CppUnitRun"; }
}
public void Reflect(RunInvokerTree tree,
RunInvokerVertex parent,
Type subjectType)
{
try
{
CppUnitFixtureAttribute fixture =
(CppUnitFixtureAttribute)(subjectType.GetCustomAttributes(true)[0]);
foreach (TestDetails test in
GetTestList(fixture.Path))
{
IRunInvoker testInvoker = new
CppUnitRunInvoker(this, fixture.Path, test.Suite, test.Test);
tree.AddChild(parent, testInvoker);
}
}
catch
{
}
}
#endregion
private IEnumerable GetTestList(string path)
{
IList testList = new ArrayList();
Process process = new Process();
process.StartInfo.FileName = path;
process.StartInfo.Arguments = "-list";
process.StartInfo.UseShellExecute = false;
process.StartInfo.RedirectStandardOutput = true;
process.Start();
string testListXml = process.StandardOutput.ReadToEnd();
process.WaitForExit();
XmlDocument xmlDoc = new XmlDocument();
xmlDoc.LoadXml(testListXml);
foreach (XmlNode node in xmlDoc.FirstChild)
{
string testName = node.InnerText;
testName = testName.Replace("::", ":");
string[] testDetails = testName.Split(':');
testList.Add(new TestDetails(testDetails[0],
testDetails[1]));
}
return testList;
}
private struct TestDetails
{
public string Suite;
public string Test;
public TestDetails(string suite, string test)
{
Suite = suite;
Test = test;
}
};
}
Unless you can spot something particularly weird in the code above, I
think I'm going to have to bite the bullet and download the MbUnit
source and step through it in a debugger. The results I'm seeing just
aren't making sense to me given the code in RunPipeStarter, and I'd
kinda like to see why that catch(Exception) block isn't detecting this
case, given the AssertionException seems to derive from Exception.
Cheers,
-A
On Feb 7, 10:48 pm, "Jeff Brown" <[EMAIL PROTECTED]> wrote:
> Thanks for the detailed description.
>
> Have you verified that your CppUnitRunInvoker is really calling Assert.Fail
> when it runs inside AutoRunner or the GUI? You could do this by launching
> the tests and attaching a debugger. Or you could log some diagnostic output
> to check.
>
> I say this because I notice you don't check the exit code of the CppUnit
> test runner process. As long as it emits some XML, the code assumes all
> went well unless a <FailedTest> element is found. It's possible that
> CppUnit failed to run the test for some other reason.
>
> For example, I also notice you don't set the WorkingDirectory of the
> process. Alternately because the StandardError stream is not being
> redirected or consumed, it may interact with the calling command shell.
> Weird things could happen.
>
> Alternately, it could be that you CppUnitRunInvoker is being decorated by
> some other run invoker that is eating the exception. I can't tell whether
> that's the case because I don't have the code for the CppUnitRun. I assumes
> it's unlikely.
>
> The only other thing I can think of is a ThreadAbortedException wreaking
> havoc. But you would probably see that in the test output just the same.
>
> What does MbUnit actually display in the test report? Does it just look
> like everything passed or are you observing some other kind of failure? Do
> the tests even appear in the report? So far you've only said that it
> doesn't look like the AssertionException gets caught. What actually
> happens?
>
> Jeff.
>
>
>
> -----Original Message-----
> From: [email protected] [mailto:[EMAIL PROTECTED] On
>
> Behalf Of Aaron Rehaag
> Sent: Thursday, February 07, 2008 8:57 PM
> To: MbUnit.User
> Subject: MbUnit Re: Assertions going unhandled by MbUnit when using
> parameterized run invokers
>
> Jeff,
>
> Long term, I'd be happy to port my solution to Gallio and contribute it to
> the MbUnit project. I'd be happy to discuss it with you more offline. In
> the meantime, I do need to find a stop-gap solution so I can use the
> technology at my workplace.
>
> Based off the code I see in RunPipeStarter, it sure seems that the
> AssertionFaliureException should be getting caught,but my tests definitely
> show otherwise. As requested, I'll provide more details.
>
> The basic idea of my solution is fairly straightforward. There are 2 major
> steps:
>
> 1) Mirror the cppunit test hierarchy in MbUnit using a custom IRun
> (CppUnitRun). Basically, we launch the cppunit test runner executable
> passing a "-list" command line option. When ran in this fashion, the test
> hierarchy is spit out in a custom XML format to cout, which CppUnitRun
> captures, and parses, adding tests to duplicate the hierarchy. This appears
> to be working correctly, so I'll spare you the details of this code.
>
> 2) We execute the tests using a custom IRunInvoker (CppUnitRunInvoker). The
> invoker launches the test runner executable once per test, passing the
> test's name as a command line option. In response, the executable spits out
> the test results (using CppUnit's built-in XMLOutputter) to cout, which are
> captured by the invoker and parsed. If the results indicate the test has
> failed, an Assert.Fail() is raised, with a message indicating details
> indicating why the test failed. Here's the code:
>
> public class CppUnitRunInvoker : IRunInvoker
> {
> public CppUnitRunInvoker(IRun generator, string path, string suite,
> string test)
> {
> _generator = generator;
> _suite = suite;
> _test = test;
> _path = path;
> }
>
> public IRun Generator
> {
> get { return _generator; }
> }
>
> public string Name
> {
> get { return (_suite + "." + _test); }
> }
>
> public bool ContainsMemberInfo(MemberInfo memberInfo)
> {
> foreach (MemberInfo member in GetType().GetMembers())
> {
> if (member.Equals(memberInfo))
> return true;
> }
>
> return false;
> }
>
> public object Execute(object o, IList args)
> {
> Process process = new Process();
>
> process.StartInfo.FileName = _path;
> process.StartInfo.Arguments = "-run " + _suite + "::" + _test;
> process.StartInfo.UseShellExecute = false;
> process.StartInfo.RedirectStandardOutput = true;
> process.Start();
>
> string testRunXml = process.StandardOutput.ReadToEnd();
> testRunXml = TrimStartingWhitespace(testRunXml);
> process.WaitForExit();
>
> XmlDocument xmlDoc = new XmlDocument();
> xmlDoc.LoadXml(testRunXml);
> XmlNode failedTests =
> xmlDoc.FirstChild.NextSibling.FirstChild;
> if (failedTests.HasChildNodes) // If the test failed.
> {
> string file = "";
> string line = "";
> string message = "";
>
> foreach (XmlNode node in
> failedTests.FirstChild.ChildNodes)
> {
> if (node.Name == "Location")
> {
> file = node.FirstChild.InnerText;
> line = node.FirstChild.NextSibling.InnerText;
> }
> else if (node.Name == "Message")
> {
> message = node.InnerText;
> }
> }
>
> MbUnit.Framework.Assert.Fail("\n\nCppUnit failure!\n \nIn "
> + file + ":" + line + "\n\n" + message); // --- BOOM! ---
> }
>
> return 0;
> }
>
> private string TrimStartingWhitespace(string str)
> {
> return str.TrimStart(new char[]{'\r','\n','\t',' '});
> }
>
> private IRun _generator;
> private string _path;
> private string _suite;
> private string _test;
> }
>
> Other than that, there's not much to it. Here's the test fixture
> itself:
>
> [CppUnitFixture("F:\\Interop\\cppunit\\debug\\unit.exe")]
> public class CppUnitSuite1
> {
> }
>
> The implementation of the CppUnitFixtureAttribute is fairly straightforward,
> so I'm not going to duplicate it here. I'll also spare you the joys of the
> test runner c++ code, but you might want the following sample test runner
> output to make sense of the XML parsing
> above:
>
> <?xml version="1.0" encoding='ISO-8859-1' standalone='yes' ?> <TestRun>
> <FailedTests>
> <FailedTest id="1">
> <Name>Suite1::Test1</Name>
> <FailureType>Assertion</FailureType>
> <Location>
> <File>f:\interop\cppunit\suite1.cpp</File>
> <Line>8</Line>
> </Location>
> <Message>assertion failed
> - Expression: false
> </Message>
> </FailedTest>
> </FailedTests>
> <SuccessfulTests></SuccessfulTests>
> <Statistics>
> <Tests>1</Tests>
> <FailuresTotal>1</FailuresTotal>
> <Errors>0</Errors>
> <Failures>1</Failures>
> </Statistics>
> </TestRun>
>
> As mentioned earlier, the Assert.Fail() raises the exception, which seems to
> go unhandled when ran through the AutoRunner, and appears to be getting
> caught by a catch block somewhere in the GUI when ran in that mode.
>
> Thoughts?
> -A
>
> On Feb 4, 3:39 pm, Jeff Brown <[EMAIL PROTECTED]> wrote:
> > Hrrm.
> > Let's see.
>
> > 1. So, I can see in RunPipeStarter that all exceptions are caught when
> > executing RunInvokers. There is no relation to arguments at all as
> > far as I can tell.
>
> > I'm not also clear where the assertion is thrown since your sample
> > below does not show one. Is it inside the test method? Does your
> > test method actually have arguments? Changing whether arguments are
> > added will effect whether the test method can actually be invoked.
>
> > I think it would help me to see more code so I can get a more complete
> > idea of what's going on.
>
> > 2. Would you be interested in porting this work to Gallio? I think
> > you'll find it much more extensible and expressive. It also runs
> > MbUnit v2, NUnit and xUnit.Net tests via an adapter. I'm currently
> > working to get the Alpha 2 release out the door.
>
> > Once that's done we can look into adding fully supported CppUnit,
> > CSUnit, MSTest adapters among others. They would probably only take 1
> > day each depending on the level of integration desired.
>
> > Jeff.
>
> > -----Original Message-----
> > From: [email protected] [mailto:[EMAIL PROTECTED]
>
> > On Behalf Of Aaron Rehaag
> > Sent: Sunday, February 03, 2008 12:07 AM
> > To: MbUnit.User
> > Subject: MbUnit Assertions going unhandled by MbUnit when using
> > parameterized run invokers
>
> > Hey everyone,
>
> > I'm trying to build a custom extension to MbUnit to execute CppUnit
> > tests and mirror the results into MbUnit test hierarchy. In order to
> > do so, I've created a custom run invoker to execute tests using a
> > cppunit runner.
>
> > Now the interesting part... when I add an Assert.Fail() statement into
> > my run invoker' Execute() method, the AssertionException thrown is
> > never caught / handled by MbUnit. As I suspected my code might be
> > flawed, I tried downloading Jay Flower's XHtml extension sample
> > solution
> > (http://jayflowers.com/WordPress/?p=88) and found the same behavior
> > running the 2.4 release. In a weird twist, it seems that if you
> > comment out the following line:
>
> > public override object Execute(object o, IList args)
> > {
> > foreach (Object Arg in this.Args)
> > {
> > // args.Add(Arg); /* Including an argument somehow makes
> > MbUnit not handle the assertion exception. */
> > }
>
> > return base.Execute(o, args);
> > }
>
> > The assertion is caught by MbUnit.
>
> > Does anyone have any insight into why MbUnit is exhibiting this
> > behavior? Am I missing some crucial logic for handling these cases
> > correctly?
>
> > Any help that can be provided would be much appreciated.
>
> > Cheers,
> > -A- Hide quoted text -
>
> > - Show quoted text -- Hide quoted text -
>
> - Show quoted text -
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"MbUnit.User" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/MbUnitUser?hl=en
-~----------~----~----~----~------~----~------~--~---
