At 11:06 PM 9/5/2007 -0700, Mikeal Rogers wrote:
On Sep 5, 2007, at 10:20 PM, Phillip J. Eby wrote:
At 08:55 PM 9/5/2007 -0700, Aparna Kadakia wrote:
It is important to remember that the framework we adopt needs to
support testing at the UI level and not within. This is more for
functional testing which covers scenarios rather than unit level
testing.
Crossing over a thread here, I think it's important to point out
that "UI level" in this context presupposes an architecture in
which there *is* such a thing.
One possible architecture refactoring is to have an "interaction
model" layer which encompasses a significant portion of what would
now be considered "UI level" code -- but *without* having any "UI"
code (i.e., graphics or I/O). Such a system would be amenable to
automated functional testing without involving any actual GUI
code. (See also my reply to John's post about transparent
persistence; this is the same visual/logical split I described there.)
I figured this was what you were getting at.
Removing the barrier of simulating end-user UI actions would make
test debugging and maintenance worlds easier. I would say it makes
test authoring easier, but if the recorded scripts framework takes
off, it's hard to compete with recorded actions vs. sitting down and
writing code.
Perhaps; of course, one of the main ideas behind doing something like
this would be to make it possible to write the tests *before* the
code exists that will be tested, as a specification of what the code
is supposed to do. There would still be a place for recorded
scenarios, of course, since some tests will be added to cover
scenarios that weren't envisioned at specification time.
I can say writing these kinds of tests would be much
easier than in the previous CATS system.
Around the beginning of the year we looked in to what automated
testing might be accomplished around this layer. I tried looking at
what automated testing might be feasible that was somewhere between
the current unittests and functional tests. What i found was that the
desktop architecture didn't seem to provide a way to break apart
these components and reliably test them in isolation, and that doing
anything beyond what the current unittests did wasn't feasible in the
short term ( the unittest coverage is already blurring the line
between functional testing and unittesting).
Yep, this is exactly the issue.
In addition, it would eliminate the issue we've had with the
functional test frameworks that are distinct from the application
itself. In an interaction-model design, the application *is* the
functional test framework, so there is no duplication of
functionality, documentation, learning, code getting out of sync, etc.
I think I understand what you mean here, but just to make sure.
You're saying that the visual components of the UI will be using the
same API that would be used for testing,
Yep.
If we do keep doing higher level automated tests ( and I imagine we
will, hopefully using the new recorded scripts framework ) there is
some value in watching them run periodically. Especially if the sole
purpose of those tests becomes catching graphical rendering problems
and we start automating the application semantics testing at other
layers.
Of course. Note, however, that in this model a functional test is a
functional test whether the GUI is hooked up or not. You could
always run them with the GUI hooked up if you wanted to watch. :)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Open Source Applications Foundation "chandler-dev" mailing list
http://lists.osafoundation.org/mailman/listinfo/chandler-dev
