On Apr 25, 2006, at 5:58 PM, Jean-Sebastien Delfino wrote:
Jim Marino wrote:
On Apr 25, 2006, at 6:28 AM, Jeremy Boynes wrote:
Generally I like this approach - I do have a couple of small
comments inline.
On 4/25/06, ant elder <[EMAIL PROTECTED]> wrote:
Create a new folder under testing called 'interop' . Interop may
not be the
best name, feel free to suggest alternatives.
interop makes sense - +1
In testing/interop there will be two other folders, 'local', and
'remote'.
testing/interop/remote will contain WS test clients that make WS
invocations
out to remote services on the Internet, so you'll need a network
connection
to run these and they'll probably be a little slow.
"client" would also be an alternative to "remote"
I assume these will be J2SE clients that can just be run with maven
and that they use WS ExternalServices to talk to the remote
services.
Will there be multiple implementations of the test clients in
different languages?
Is there a reason you were thinking about why these wouldn't be
JUnit-based or is that considered J2SE?
testing/interop/local will include implementations of the WS
services used
by all the clients in the remote folder. It will also have
clients to invoke
those services but they'll just be extending the client code in
the remote
folder to override the endpoint used to be the local service but
without
requiring too much code duplication of the client code.
Perhaps "server" as an alternative to "local"
Will we have to implement these services or are there existing
versions we can use?
Are there other non-SCA clients that can be used to call them?
Within the local and remote folders there will be separate
projects for each
set of tests, so projects for whitemesa, aspnetround3, amazon etc.
These tests are using the actual SCA client APIs, they're not
gong to be
fudging things like the existing WS test in the tomcat
integration tests.
The testing/interop/local services need to be deployed and run
in tomcat to
work, so that will be done by the existing testing/tomcat stuff.
I haven't
quite though this part through yet, I guess that means having
testing/tomcat
get all the testing/interop/local sevices deployed and adding to
testing/tomcat an interop test project which calls all the
testcases in
testing/interop/local. It doesn't seem perfect to require having
testing/tomcat/interop as well as testing/interop, is there a
better/easier
way to do this?
This should not depend on a special Tomcat installation but
should be
deployable to our normal Tomcat distribution (or any other distro we
do) - that way we know the distro that users would actually download
is being tested.
+1 on this particularly as we add more host environments
I'm not sure you really need the special clients here - just run the
server and then run the real clients (from "remote") against it.
That
way we know our real interop clients talk to our real server with no
"fudging."
I also like this given that if we test against our own servers
it's not interop
So with that config you can run all the testing/interop/local
tests to test
out the Tuscany WS entryPoint and externalService function. Or
you can run
all the testing/interop/remote tests to verify interaction with
real
non-Tuscany systems.
Interop is all about working with non-Tuscany systems. I think
having
the ability to run their clients against our server is very
important.
Ant or someone more familiar with Axis, I assume there are some
mechanisms at Apache for setting this type of thing up?
For now I wont even mention the question of when any of these
tests should
get run as part of a build process :-)
Given the dependencies (e.g. on remote servers, other people's
clients) then I don't think these should be run as part of the
build.
This should be done in a controlled environment on a periodic basis,
and at least once with any release candidate.
Any comments or suggestions?
Thanks
--
Jeremy
+1 from me on all of this.
I am assuming that the client side tests are Junit tests.
I like Jeremy's proposal to name these client / server.
I agree that we need to run these tests on top of our actual distro.
Two questions:
- can we run this daily with continuum?
- can we have a single client test suite and externalize the
endpoints that it talks to, with a Maven configuration for example?
(this would be simpler than the initial template that I had checked
in, which had common test code but still required different test
cases to talk to our server and the remote test server)
I'd like to add that I think we need unit and "integration" testing
run as part of the build which covers some of the expected interop
behavior. In other words, we should have good enough coverage in our
build where a basic interop test (e..g not serializing something
properly) surprises us. I was thinking the integration tests would
not deploy to a servlet container but would use mock components and
programmatically verify messages flowing in and out of entry points
and external services. It would also be nice to have a generic unit/
integration test harness that could be run using different ws
bindings such as Axis/Celtix. Does this make sense to people?
Dan, you mentioned on chat that Celtix has a mechanism for doing
integration testing. I raised this question a several times on the
list (we have a lot of duplicate "mock" code". Could you describe
what you guys have done and if you think it is appropriate for us? I
think you are running into some problems related to this as you write
unit tests for the Celtix binding since many of the mock classes and
factories will need to be copied.
Thanks,
Jim
--
Jean-Sebastien
