Quoting Steve Reinhardt <[email protected]>:
> On Sun, Mar 7, 2010 at 11:50 PM, Gabe Black <[email protected]> wrote:
>>
>>> (Practically speaking, do we
>>> support anything other than linux on any other ISA?)
>>>
>>
>> Only supporting Linux doesn't mean the OS never changes. It would be
>> nice to be able to test both 32 and 64 bits of Linux in both FS and SE,
>> or even different configurations of each. This is especially true in x86
>> where the ISA actually changes significantly, as does the the ABI in SE
>> mode. As we've found in the past with x86 (unless I'm confused again)
>> testing 64 bit binaries didn't mean 32 bit support wasn't rotting out.
>
> OK, that's a good thing to keep in mind. I don't think it changes my
> opinion that it's overkill to make the OS a first-class parameter in
> the testing scheme.
Yeah, I didn't necessarily think they were in conflict, I just wanted
to make sure we accounted for that sort of thing this time around.
>
>> I think generally we want to make sure we keep things flexible, but at
>> the same time we don't want it to turn into mush where it gets
>> cumbersome to do common things. I don't know the best way to actually do
>> that though.
>
> That sounds like a pretty generic summary of software interface design :-).
Yes :). What I was getting at, though, is that we don't want to force
tests to fit a certain mold (all linux tests are equivalent, all tests
care about ISA, etc.) but at the same time not to go to totally
anarchy where the majority of tests that -do- fit in that mold need a
bunch of extra gunk to get to a simple result. The idea of building
the same classification structure we've had implicitly but now
explicitly for each test is what I'm worried about. Also we'd want to
be able to escape the system every now and then when a test almost
fits but needs a few special tweaks.
>
>> This is more wishful thinking than anything else, but wouldn't it be
>> nice if our regression tests also included tests that exercised some
>> feature or component more comprehensively and specifically? Sure,
>> booting Linux is very important and touches a lot, but it doesn't touch
>> everything and it takes a while to do it. I still can't do anything like
>> this because of conflict of interest, but it would be nice to have.
>
> I'll second this, but my impression is that that's an issue of just
> putting in the effort to construct those tests, rather than any
> structural problems with the test framework. Am I wrong?
No, you're right. It would be nice to consider that possible future in
the design which is why I brought it up.
>
>> -If-
>> that ever came to be, groups would be nice because they could define a
>> set of tests to run to test a certain piece of functionality. I also
>> think you're onto something, Steve, where you define "groups"
>> dynamically as a collection of traits rather than an otherwise arbitrary
>> grouping of tests. The hard part is picking the set of traits to keep
>> track of and actually measuring them.
>
> Maybe instead of (or in addition to) the numeric tags I proposed, each
> test could also have an optional tuple of strings, e.g., ('quick',
> 'mem'), and then when you run tests you could supply a string and it
> would run all the applicable tests with that tag. Sort of like hg
> qselect. Of course you'd still want 'all' to run everything (so the
> more specific tags are optional).
>
Sure, that could work.
Gabe
_______________________________________________
m5-dev mailing list
[email protected]
http://m5sim.org/mailman/listinfo/m5-dev
