Is there any documentation on how to run the current unit tests? I notice that there's a unit test directory in the X86 build folder, but nothing inside is executable. I do not see a way of generating output from SCons related to unittests either.
From the src/SConscript file, I see that the unittests should get created inside the makeEnv function, but it doesn't appear that this does anything (at least for X86). -Brandon -----Original Message----- From: gem5-dev [mailto:[email protected]] On Behalf Of Andreas Sandberg Sent: Wednesday, November 22, 2017 3:35 AM To: gem5 Developer List <[email protected]>; Gabe Black <[email protected]> Subject: Re: [gem5-dev] [EXT] Re: testing framework One issue that is preventing me from running the unit tests is that they require separate scons targets. One solution that I was considering at one point was to make the unit tests available as Python exports in the main build. This would allow us to run a magic config that calls unit tests instead of running the simulator proper. I think it would work for all(?) current unit tests, but I don't know how it would interact with other test unit test frameworks. Cheers, Andreas On 21/11/2017 01:37, Gabe Black wrote: > What I would minimally like to see is for people to run the unit tests > that already exist. Clearly that hasn't been happening since one of > them hasn't compiled since 2014 (now fixed). We need our tests to test > something specific, say whether or not it worked (not leave > interpretation up to the reader), and for people to actually run them. > > It's not at all clear what to do to actually build the unit tests from > scons, or how to run them, or what their results mean. I'm going to > try cleaning these up a bit. Also, they shouldn't be something that > lives off on the side or in their own directory, out of sight and out > of mind, they should be alongside the code they test. > > It might even make sense to associate a unit test with a particular > source file and make compilation fail if the unit test doesn't pass. > That might be a bit overboard, but would at least ensure that the tests get > run. > > Gabe > > On Mon, Nov 20, 2017 at 3:14 PM, Potter, Brandon > <[email protected]> > wrote: > >> I also support adding unit tests (and possibly a code coverage >> checker to identify weak points). Ideally, we'd add unit tests for >> the entire simulator, but we should probably put something into place >> for new code requiring some type of unit tests. Otherwise, only a >> subset of the submissions will contain unit tests. >> >> Regards, >> Brandon >> >> -----Original Message----- >> From: gem5-dev [mailto:[email protected]] On Behalf Of Jason >> Lowe-Power >> Sent: Monday, November 20, 2017 11:22 AM >> To: gem5 Developer List <[email protected]> >> Subject: Re: [gem5-dev] [EXT] Re: testing framework >> >> I'm all for unit tests. GTest seems fine, but I'm not a huge fan of >> its syntax. Although, that shouldn't stop us from having unit tests!! >> >> What's the state of getting whimsy pushed in? From my side, it's >> ready to go. The only thing on my to do list that I would have liked >> to finish up before pushing is better integration with the current test >> infrastructure. >> It would be good for whoever is running with the old test >> infrastructure to be able to run both the old-style and new-style tests with >> one command. >> Unfortunately, I don't have the time to dig into this for the >> foreseeable future. >> >> Jason >> >> On Fri, Nov 17, 2017 at 3:19 PM Paul Rosenfeld (prosenfeld) < >> [email protected]> wrote: >> >>> Even without the complexities of setting up SystemC+gem5 it's pretty >>> painful to "unit" test SystemC because the SystemC runtime has a lot >>> of global state that comes along with it (for example: you can't "uncall" >>> sc_start() to go back and re-do elaboration with different modules). >>> The only (not very elegant) workaround we have discussed for unit >>> testing SystemC modules is to just to build each set of related >>> SystemC unit tests into their own independent GTest binaries. >>> >>> On the whole, though, GTest is pretty great. >>> >>> -----Original Message----- >>> From: gem5-dev [mailto:[email protected]] On Behalf Of Gabe >>> Black >>> Sent: Friday, November 17, 2017 5:51 PM >>> To: Matthias Jung <[email protected]> >>> Cc: gem5 Developer List <[email protected]> >>> Subject: [EXT] Re: [gem5-dev] testing framework >>> >>> As far as a high level framework, I think whimsy should help there >>> since it's a bit more decoupled and will let you run arbitrary >>> things as tests, at least as far as I can tell. Since the systemC >>> integration is just that, an integration of two systems, I don't >>> think unit tests would be as applicable generally speaking. You >>> could have one for some things though, like the ports that translate >>> between gem5's and >> systemC's memory protocol. >>> GTest claims to be BSD, so the license should be compatible. >>> >>> Gabe >>> >>> On Fri, Nov 17, 2017 at 10:14 AM, Matthias Jung >>> <[email protected]> >>> wrote: >>> >>>> Hey, >>>> >>>> It would be also nice if we have something that we could also use >>>> for test the SystemC stuff. Right now this is not possible with the >>>> current framework. Because we have to build gem5 without python as >>>> a library for the SystemC stuff... and then build again with >>>> linking of >>> the gem5 library. >>>> Regards >>>> Matthias >>>> >>>>> Am 17.11.2017 um 17:43 schrieb Andreas Sandberg < >>>> [email protected]>: >>>>> I'm generally in favour of switching to an existing unit test >>> framework. >>>>> I haven't surveyed the C++ test framework landscape, but judging >>>>> by the users (Chromium, LLVM, ...) GTest seems pretty solid. >>>>> >>>>> As far as I can tell, GTest is all BSD. Is that correct? >>>>> >>>>> Cheers, >>>>> Andreas >>>>> >>>>>> On 16/11/2017 05:04, Gabe Black wrote: >>>>>> Hi folks. How is the test framework integration coming along? Is >>>>>> there anything I can do to help? >>>>>> >>>>>> Also, looking at the whimsy documentation and talking to some >>>>>> other >>>> folks >>>>>> at Google, it looks like the framework is good at running tests, >>>>>> but doesn't itself really implement, for instance, unit tests >>>>>> which have >>>> their >>>>>> own C++ main function, etc. >>>>>> >>>>>> Does anyone have an opinion about using the google gtest library >>>>>> for writing C++ unit tests? I've only worked with it a little bit >>>>>> superficially, but so far it seems pretty nice and seems like it >>>>>> would >>>> fit >>>>>> into whimsy fairly well. >>>>>> >>>>>> Gabe >>>>>> _______________________________________________ >>>>>> gem5-dev mailing list >>>>>> [email protected] >>>>>> http://m5sim.org/mailman/listinfo/gem5-dev >>>>> IMPORTANT NOTICE: The contents of this email and any attachments >>>>> are >>>> confidential and may also be privileged. If you are not the >>>> intended recipient, please notify the sender immediately and do not >>>> disclose the contents to any other person, use it for any purpose, >>>> or store or copy the information in any medium. Thank you. >>>>> _______________________________________________ >>>>> gem5-dev mailing list >>>>> [email protected] >>>>> http://m5sim.org/mailman/listinfo/gem5-dev >>>> >>> _______________________________________________ >>> gem5-dev mailing list >>> [email protected] >>> http://m5sim.org/mailman/listinfo/gem5-dev >>> _______________________________________________ >>> gem5-dev mailing list >>> [email protected] >>> http://m5sim.org/mailman/listinfo/gem5-dev >> _______________________________________________ >> gem5-dev mailing list >> [email protected] >> http://m5sim.org/mailman/listinfo/gem5-dev >> _______________________________________________ >> gem5-dev mailing list >> [email protected] >> http://m5sim.org/mailman/listinfo/gem5-dev >> > _______________________________________________ > gem5-dev mailing list > [email protected] > http://m5sim.org/mailman/listinfo/gem5-dev IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. _______________________________________________ gem5-dev mailing list [email protected] http://m5sim.org/mailman/listinfo/gem5-dev _______________________________________________ gem5-dev mailing list [email protected] http://m5sim.org/mailman/listinfo/gem5-dev
