I'd just like to second Rob here. The work you're doing Heath is awesome!
On Jun 12, 2014 9:03 AM, "Rob Mensching" <r...@firegiant.com> wrote:
> Heath,
>
>
>
> The pull request looks really good. The infrastructure improvements you
> made seemed very much in line with the changes I made to initially get
> Xunit to support our basic integration test needs.
>
>
>
> This is an awesome step and we can keep improving from here.
>
>
>
> The root issue was that Xunit is a unit test framework, not an integration
> test framework. We’re stretching it a bit to make it handle the integration
> tests for Burn. However, all the additions we’ve added have seemed
> reasonable. So I’m still pretty encouraged by the benefits of being able to
> use Xunit as our unit test and integration test framework.
>
>
>
> Anyway, thanks for the huge update of the test framework. Will get this in
> very soon.
>
>
>
> _______________________________________________________________
>
> FireGiant | Dedicated support for the WiX toolset |
> http://www.firegiant.com/
>
>
>
> *From:* Heath Stewart [mailto:hea...@outlook.com]
> *Sent:* Wednesday, June 11, 2014 2:20 AM
> *To:* Windows Installer XML toolset developer mailing list
> *Subject:* Re: [WiX-devs] Test overhaul
>
>
>
> Thanks. I'm aware of how xunit works (having read articles and through
> their code), but we also need tear-down methods to clean up, especially,
> failed install tests.
>
>
>
> The disposable pattern works great, but requires a LOT more source
> changes. I've already had to touch hundreds of lines of code across over a
> hundred source files. Once I get this checked in, I'm happy to let someone
> else take a stab at improving it.
>
>
>
> For right now, my main goal is to get the test system working again. I
> don't feel comfortable checking in code without tests. Didn't (and don't)
> do it with our internal “fork” and don't want to start with the external
> one either. Having all tests pass used to be a requirement before checking
> in code.
>
>
>
> - Heath from his Surface Pro
>
>
>
> *From:* Bert Huijben <b...@qqmail.nl>
> *Sent:* Wednesday, June 11, 2014 2:19 AM
> *To:* Windows Installer XML toolset developer mailing list
> <wix-devs@lists.sourceforge.net>
>
>
>
> [Happy long time wix user… Mostly just reading on this list by sending it
> to a mailbox, but if I can offer any help…]
>
>
>
> Unlike nunit, xunit initializes a new class instance before running any
> test, so you can just use the constructor to initialize things that you
> need in your test.
>
>
>
> See http://jamesnewkirk.typepad.com/posts/2007/09/why-you-should-.html
> for the best explanation on why you normally shouldn’t use setup and
> teardown as in NUnit…
>
>
>
>
>
> Personally I would recommend running tests inside an explicit using(),
> with an appropriate helper class if you need some explicit state only
> during a test. This documents the test and its context in one place.
>
>
>
> Bert
>
>
>
> *From:* Heath Stewart [mailto:hea...@outlook.com <hea...@outlook.com>]
> *Sent:* woensdag 11 juni 2014 05:53
> *To:* WiX toolset developer mailing list
> *Subject:* Re: [WiX-devs] Test overhaul
>
>
>
> I've changed quite a bit of the infrastructure as well, but xunit doesn't
> support initialization and tear-down and a lot of other things required by
> many of our tests. I can rename the attribute back but I wouldn't say xunit
> is a solid replacement.
>
> Sent from my Windows Phone
> ------------------------------
>
> *From: *Rob Mensching <r...@firegiant.com>
> *Sent: *6/10/2014 12:11 PM
> *To: *WiX toolset developer mailing list <wix-devs@lists.sourceforge.net>
> *Subject: *Re: [WiX-devs] Test overhaul
>
> I only moved a few Burn projects to Xunit to prove that it could be done
> (added NamedFactAttribute and other things). I left the others still
> running against MSTest. It’s fantastic if you finished moving the other
> tests.
>
>
>
> However, please don’t change Xunit to look like MSTest. “NamedFact” was
> named that way to model the fact that Xunit calls tests “Facts”. Also, did
> we use priority and description?
>
>
>
> Also, let’s wait until Xunit goes to 2.0 before moving to it.
>
>
>
> There was a lot of nasty test code and test infrastructure to clean up and
> simplify in there. I totally understand if you only moved the test code and
> didn’t scrub infrastructure. <smile/>
>
>
>
> _______________________________________________________________
>
> FireGiant | Dedicated support for the WiX toolset |
> http://www.firegiant.com/
>
>
>
> *From:* Heath Stewart [mailto:hea...@outlook.com <hea...@outlook.com>]
> *Sent:* Monday, June 9, 2014 9:40 AM
> *To:* Windows Installer XML toolset developer mailing list
> *Subject:* Re: [WiX-devs] Test overhaul
>
>
>
> It was the changeover that they seemed to all (well, mostly) break. Xunit
> doesn't work the same way as MSBuild or NUnit so any tests that require
> test-case initialization and tear-down would fail. With some extensibility
> I was able to put that behavior back, for the most part. Nearly finished.
>
>
>
> The really, super annoying thing has been having to walk around xunit's
> sticks in the sand, regarding user messages for Asserts (or lack thereof)
> and that even those Asserts that support user messages don't support params
> parameters like String.Format, so I have to add that in everywhere to
> maintain the existing behavior.
>
>
>
> All tests but WixTests are building and almost all of those are passing.
>
>
>
> I also took the opportunity with the extensibility to skip tests that are
> runtime tests when runtime tests aren't enabled rather than fail, skip
> runtime tests that require elevation (the assumed default - tests can
> declare themselves NonPrivileged), and skip tents that are 64-bit specific
> when not running under a 64-bit process. These are effectively trait
> attributes, and will be proper TraitAttribute-derivatives once Xunit 2.0
> releases (or I can now if we want to take a pre-rely, doesn't really change
> much, though).
>
>
>
> Since I had to define a custom attribute anyway, I renamed NamedFact to
> TestMethodAttribute, and added traits PriorityAttribute,
> DescriptionAttribute, RuntimeTestAttribute, and
> Is64BitSpecificTestAttribute.
>
>
>
> I'll have a pull request in a day or two. All that's left in the WixTests
> project is wrapping a bunch of user messages in String.Format. I have 167
> instances to fix up yet where regex-based replacement may not be of much
> help. I'm considering options for automation, including Roslyn at least to
> help automate the changes.
>
>
>
> - Heath from his Surface Pro
>
>
>
> *From:* Hoover, Jacob <jacob.hoo...@greenheck.com>
> *Sent:* Monday, June 9, 2014 6:27 AM
> *To:* Windows Installer XML toolset developer mailing list
> <wix-devs@lists.sourceforge.net>
>
>
>
> I don’t know when they “all broke”, but I do know I had tests failing when
> I was trying to validate changes I had submitted. Some of them had to do
> with case sensitivity of paths, but I can’t remember the others. I do
> think at one point in time I had made an attempt to fix it, but with the
> changeover in testing frameworks I never submitted a pull request.
>
>
>
>
>
> *From:* Heath Stewart [mailto:hea...@outlook.com <hea...@outlook.com>]
> *Sent:* Monday, June 09, 2014 2:26 AM
> *To:* Windows Installer XML toolset developer mailing list
> *Subject:* [WiX-devs] Test overhaul
>
>
>
> I've nearly got the fully xunit-based test system working again. I felt it
> important to get it working again whilst checking in all the new code, some
> of which are new tests. I was able to keep some of the same behavior, like
> something similar to TestInitialize and TestCleanup attributed methods.
> Once xunit 2.0 releases (I've got some TODOs in the code as reminders) I
> can beef it up more, but xunit's current extensibility gave me some new
> options. Just trying to get the TestDataDirectory working, which points to
> %WIX_ROOT%\tests\data without relying on the environment variable.
>
>
>
> Perhaps this is more directed at Rob or Bob, but I'm curious if there were
> any tests failing before they all broke. I recall that some of the burn
> tests were failing and we turned them off in the internal fork, but all the
> tests in WixTest.dll are working.
>
>
>
> - Heath from his Surface Pro
>
>
>
>
> ------------------------------------------------------------------------------
> HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
> Find What Matters Most in Your Big Data with HPCC Systems
> Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
> Leverages Graph Analysis for Fast Processing & Easy Data Exploration
> http://p.sf.net/sfu/hpccsystems
> _______________________________________________
> WiX-devs mailing list
> WiX-devs@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/wix-devs
>
>
------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
WiX-devs mailing list
WiX-devs@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/wix-devs
