Re: [lldb-dev] Separating test runner and tests

2015-12-11 Thread Pavel Labath via lldb-dev
Sounds like a reasonable thing to do. A couple of tiny remarks: - when you do the move, you might as well rename dotest into something else, just to avoid the "which dotest should I run" type of questions... - there is nothing that makes it obvious that "engine" is actually a "test running

Re: [lldb-dev] Separating test runner and tests

2015-12-11 Thread Zachary Turner via lldb-dev
Yea wasn't planning on doing this today, just throwing the idea out there. On Fri, Dec 11, 2015 at 9:35 AM Todd Fiala wrote: > I'm fine with the idea. > > FWIW the test events model will likely shift a bit, as it is currently a > single sink, whereas I am likely to turn it

Re: [lldb-dev] Separating test runner and tests

2015-12-11 Thread Todd Fiala via lldb-dev
I like it. On Fri, Dec 11, 2015 at 9:51 AM, Zachary Turner wrote: > Yea wasn't planning on doing this today, just throwing the idea out there. > > On Fri, Dec 11, 2015 at 9:35 AM Todd Fiala wrote: > >> I'm fine with the idea. >> >> FWIW the test events

Re: [lldb-dev] Separating test runner and tests

2015-12-11 Thread Todd Fiala via lldb-dev
Unittest. Comes with Python. On Fri, Dec 11, 2015 at 11:07 AM, Zachary Turner wrote: > Presumably those tests use an entirely different, hand-rolled test running > infrastructure? > > On Fri, Dec 11, 2015 at 10:52 AM Todd Fiala wrote: > >> One thing I

Re: [lldb-dev] Separating test runner and tests

2015-12-11 Thread Todd Fiala via lldb-dev
It just requires running the test file as a python script. The runner is fired off like this: if __name__ == "__main__": unittest.main() which is typically added to the bottom of all test files so you can call it directly. -Todd On Fri, Dec 11, 2015 at 11:12 AM, Todd Fiala

Re: [lldb-dev] Separating test runner and tests

2015-12-11 Thread Todd Fiala via lldb-dev
The tests end up looking substantially similar to our lldb test suite tests, as they were based on unittest2, which is/was a relative of unittest that now lives in Python. The docs for unittest in python 2.x have generally been accurate for the unittest2 lib we use. At least, for the areas I

Re: [lldb-dev] Separating test runner and tests

2015-12-11 Thread Todd Fiala via lldb-dev
I think we can do this, and I'd like us to do this unless it's proven to break something we're not aware of. I think you did some research on this after we discussed last, but something (maybe in the decorators) didn't just work. Was that right? On Fri, Dec 11, 2015 at 11:18 AM, Zachary Turner

Re: [lldb-dev] Separating test runner and tests

2015-12-11 Thread Todd Fiala via lldb-dev
Okay. Sounds like something we can work around one way or another, either by introducing the correct exception name for unittest, or introducing our own if we need to do so. On Fri, Dec 11, 2015 at 11:22 AM, Zachary Turner wrote: > If I remember correctly it was in the way

Re: [lldb-dev] Separating test runner and tests

2015-12-11 Thread Zachary Turner via lldb-dev
Presumably those tests use an entirely different, hand-rolled test running infrastructure? On Fri, Dec 11, 2015 at 10:52 AM Todd Fiala wrote: > One thing I want to make sure we can do is have a sane way of storing and > running tests that test the test execution engine.

Re: [lldb-dev] Separating test runner and tests

2015-12-11 Thread Todd Fiala via lldb-dev
One thing I want to make sure we can do is have a sane way of storing and running tests that test the test execution engine. Those are tests that should not run as part of an "lldb test run". These are tests that maintainers of the test system run to make sure we're not breaking stuff when we

[lldb-dev] Separating test runner and tests

2015-12-10 Thread Zachary Turner via lldb-dev
Currently our folder structure looks like this: lldbsuite |-- test |-- dotest.py |-- dosep.py |-- lldbtest.py |-- ... |-- functionalities |-- lang |-- expression_command |-- ... etc I've been thinking about organizing it like this instead: lldbsuite |-- test