Hi,
I was thinking a lot about this whole unit testing thing, after the
postings last week and I finally concluded for me that it might be
worth to further investigate this area and share some of my ideas
with you all.
Before going deeper in this topic, I will first emphasis why the
existing solution don’t satisfy my needs and some other approach is
desirable. Here are some (not all) features a testing framework in
OSGi should offer from my point of view.
- distributable as a pure OSGi bundle
- not bigger than a few KB (except the JUnit classes)
- different output possibilities based on test not only on the runner
or environment (like XML, Database, custom,...)
- service based
- compatible to JUnit 3 & 4
- dynamic and configurable (distinguish between different testing
methods, like “run only regression test, no installation or update”,
run every day at 12pm all xyz tests)
- based on a standard or recommendation (f.i. naming conventions)
As far as I understood, all approaches, which are available out there
don’t fulfill most of the mentioned criteria. The eclipse way for
instance is pretty full blown and lacks in terms of services and
custom output (Please correct me if I am wrong). By the way, does
anyone know, if it is possible to run the not gui based testing stuff
provided by eclipse, but based on another OSGi framework like felix,
knopflerfish or other commercial ones? This would be desirable too,
if a customer is looking for vendor support for instance (see sample
later).
Some may say, ok - nice, but why do you need all that. Finding use
case for this requirements, is no problem if you just look into 24/7
application (attach testframework and after testing, remove it
again), server based applications with user provided
“addons” (testing the user submissions before integration or usage)
or the development statistics based on a regression tests that have
to be stored in a database every day. With a smart and pure OSGi
based framework, this can be achieved pretty easily.
I think, testing is in many areas pretty much underestimated. Of
course, there are attempts by Beck, Gamma, Link and all the other
agile programmers to introduce things like test first, but to be
honest, I don't know any one personally how really does it the way
they claim it. Especially if you are in a kind of time pressure in
your project, depending on the company you are in, only the features
implemented count and not the tests you wrote to proof the
functionality (of course this is not always the case, but it is still
not something uncommon). I think this might be a reason, why there is
still no real satisfying solution for testing out there. If it is
possible to establish a sort of infrastructure for unit testing, it
might solve a lot of issues which we are faced almost every day, like
reuse and seamlessly integration of new tests into an existing test
framework.
This is probably a long shot, but I was wondering, if it might be an
interesting topic for a short talk at the Eclipse Con (Don’t get me
wrong, this is not intended to be a proposal at all, more a general
question). Things like possible scenarios for such an testing
environment, including the business values might be pointed out
there. Based on such a talk, it think, it could be even possible to
set up a BOF discussion about topics like:
- which services should be provided and how (naming conventions)
- how to distribute test code (fragment, extra bundle, two different
bundles,...)
- architecture ( testrunner, testresultconsumer, APIs, ...)
- core features
- possibilities how to align with the OSGi spec, like a OSS bundle, a
spec like recommendation, ...
The situation would be perfect, to get many people together and
participate in a satisfying approach for a wide range of developers.
Where else can you get that fast so much feedback and impulses?
As I mentioned above, this is a long shot and not intended to be
anything more than a starting point for a discussion. I think at the
moment testing is something, which most of us do, but the reusability
of test code is almost not existing in many cases and reuse of whole
test environments even in companies is pretty poor elaborated. With
relatively few amount of work it should be possible to at least come
up with some general recommendations concerning naming and API
issues, to enable a OSGi like reuse of components.
I am looking forward to hear your comments on this.
Thanks,
Mirko
--
Mirko Jahn
e-mail: mirkojahn {_at_} gmail.com
skype: sooja-mj
_______________________________________________
OSGi Developer Mail List
osgi-dev@bundles.osgi.org
http://bundles.osgi.org/mailman/listinfo/osgi-dev
