I think I'm going to make 2 test groups for now... one to test testng
and one to test the junit decorator bits... then we can see which
works better.
I've been playing with console-testsuite today using testng and it
works well... nice to have tags to specify what methods to run before
the suite and after the suite... I think we might have to do some
magic with JUnit to make that work.
I also hooked up selenium to be able to use some custom extensions...
specifically the ability to removeCookie(), but we can add more as
needed.
And, I changed the way that the server starting goals work, so that
you can run them on the command-line with maven to start them up,
which is much better to help develop. For example you can:
mvn selenium:start &
mvn geronimo:start &
...
mvn geronimo:stop
This will default to starting up Jetty + J2EE, but if you want to
test with Tomcat + J2EE, you can:
mvn geronimo:start -DassemblyId=tomcat
...
mvn geronimo:stop -DassemblyId=tomcat
jetty-minimal and tomcat-minimal can also be used with artifactId,
though until I get a better way to tell when the server is fully
started via JMX, background mode for these assemblies won't work.
They can still be background'd for testsuite usage, though needs to
have the background flag set to true.
The geronimo:start goal is also smart enough to detect when a new
assembly is available, so the next time you run it it will re-
install... else it will just reuse the previously extracted assembly.
Probably need to expose a command-line param to allow the basedir to
be changed... for windows folks... since the default is to unzip
under console-testsuite/target/, which I'd guess is probably going to
be a tad to long for windows.
Plugin docs have been updated if anyone is interested:
http://geronimo.apache.org/maven/server/maven-plugins/geronimo-
maven-plugin/index.html
http://geronimo.apache.org/maven/server/maven-plugins/selenium-
maven-plugin/index.html
Finally... geronimo:start and stop are usable from the main tree as
well, which makes it nice and easy to build and run the server:
mvn install
mvn geronimo:start -Dverbose=true
* * *
So I'm going to svn cp console-testsuite/basic to console-testsuite/
basic-junit and then apply your patch to that tree, and then commit
the changes to basic that I have done to use testng, plus some other
support/helper classes.
Oh, and I did get some xpath validation working... but the xpath
string is horrible... I used XPather (http://xpath.alephzarro.com/)
to select some text and give me the xpath. It did work but ends up
like:
<snip>
assertEquals("Server Info", selenium.getText(
"xpath=/html/body/[EMAIL PROTECTED]'rootfragment']/tbody/tr[2]/td/
table/tbody/tr[2]/td[4]/table/tbody/tr[1]/td/table/tbody/tr[1]/td/div/
table/tbody/tr/td[2]/table/tbody/tr/td[1]/strong"));
</snip>
We may need to insert some id's into the console's pages to allow us
to root to the major panels.
--jason
On Sep 2, 2006, at 6:28 AM, Bill Dudney wrote:
Hi Jason,
This is very cool indeed, thanks for putting the prototype
together. I just submitted a patch that makes firefox pop up only
once to run all the tests (it actually pops up twice, once during
selenium.start() and then once for the test suite but the first
time is for a split second).
http://issues.apache.org/jira/browse/GERONIMO-2374
It uses a JUnit TestDecorator to run setup only once for the whole
suite.
IMO until we move to JDK 5 its not a great idea to move to TestNG,
you will get some benefits for sure but the source has to be parsed
at runtime to get at the meta-data (i.e. the info about the tests).
Krufty IMO.
The patch has a comment about invalid XHTML. I believe the invalid
XHTML part of the console is preventing the XPath find from working.
TTFN,
-bd-
On Sep 1, 2006, at 7:46 PM, Jason Dillon wrote:
I've checked in a new top-level testsuite module and a few new
plugins to support it. This is only the start and I expect it to
change over the next few weeks (er maybe months) as momentum
starts to pick up.
I looked into Cargo, and while I think we should eventually use it
to start/stop the server... asis now its broke. We need to
provide a very simple and consistent way for Cargo to invoke some
operations via JMX, which I hinted to in previous emails. Once
that is done, we can patch Cargo and hopefully get that committed
to support G... but in the mean time I whipped up a simple start &
stop mojo that uses Ant to start and stop the server. Its very,
very crude and we need to fix stuff like logging output etc.
I also played around a little with Selenium to make some tests for
the console. It wants a special server process started, so I
created a selenium plugin which currently only starts that server
in... well an icky way, but should work for the moment. Need to
ask the m2 folks how to do this better.
I created a console-testsuite module, which sets up the G server
and Selenium server in pre-integration-test, and then uses the
maven-invoker-plugin to invoke the basic module to actually run
the tests. Only one test class right now, SimpleLoginTest, which
does just that... logs in, logs in and then out and then another
that logs in and clicks some links.
I've got it all running from one mvn command and so far, on my
mac, it works fine. I'd imagine due to the location of the
assembly that Windows will not be able to run this... we'll need
to fix this later. But those of you lucky enough to have a Linux
box or a mac, you should be able to:
mvn clean && mvn install
cd testsuite
mvn
(the testsuite is not hooked up to the default build)
And you should see Firefox popup a few times as Selenium goes
about clicking on stuff. Right now its only really doing
validation of the console page's titles... but if everything
works, then the invoker should not complain and the build should
complete.
A bunch more needs to be done... like creating a common test
support class for selenium tests... and probably switching these
tests to use testng, so that the selenium setup is done once, not
for each test... as it does now... so expect Firefox to start and
stop a few times. We need to have some helpers to pick out the
console contents easily so they can be validated... I read that
selenium supports xpath or something, but I have not actually
tried any of that.
This is just a very, very crude proof of concept... needs to have
reporting features added, capture logs, blah, blah, blah.
Anyways, I committed this so we have a common base to work patches
off of.
One thing to note about the Selenium IDE, is that sometimes it
does not put into a waitForPageToLoad(), like if you record a
login to the console... you need to manually add that wait after
the button is clicked or it will hose the test.
I'm not married to any of this either... I just wanted something
that worked... or kinda worked to show that its not going to be
that hard to make some rich integration tests.
So, comments, suggestions, patches... all welcome.
--jason
