Comments inline.
Simon Laws wrote:
I think there is a difference in the way that we are looking at these
samples.
- We could consider that the samples are purely to show SCA running in
various scenarios and nothing more, i.e. anything extra such as sample
tests
or extra build steps is superfluous.
- Alternatively we could consider the samples to be a demonstration
of, not
only the function of the SCA runtime, but of how you might start to build
software that uses SCA, for example, you will likely need some tests
and you
will want to be able to integrate the sample into your IDE of choice.
I prefer that approach. SCA is a programming model for application
developers. Samples show how you use develop an application. Test is an
important part of development :) So I think we should have Junit test
cases as part of the sample code.
I have been thinking along the latter lines and stepping back I'm
thinking
maybe we need to do more on testing rather than less, i.e. include a
JUnit
target in the ant script. We also need some more IDE detail and I've
already
marked this as a TODO in the top level samples README.
An Ant test target would be nice to have, but not mandatory for this
release IMO. Even with an Ant target I think we should keep the current
structure, which I believe follows common development practice: tests in
a separate directory, and not packaged in target JARs or a binary distro.
Anyhow I think this is why we seem to be coming at this from different
angles. So probably need others opinions to see which way to go.
Some more comments in line taking this into account.
On 5/11/07, Simon Nash <[EMAIL PROTECTED]> wrote:
Thanks for the detailed review and comments. My responses are inline.
Simon
Simon Laws wrote:
> Hi, I was thinking about this problem the other way round
>
> At the moment the binary distribution builds with the following
structure
> (taking binding-echo and simple-bigbank as two examples)
>
> docs/
> lib/
> modules/
> samples/
> binding-echo
> src/
> main/
> extension sample code
> test/
> java/
> echo/
> EchoBindingClient.java
> EchoReferenceTestCase.java
> EchoServiceTestCase.java
> simple-bigbank/
> src/
> main/
> java/
> bigbank/
> client/
> BigBankClient.java
> test/
> java/
> bigbank/
> BigBankTestCase.java
>
> So there are a number of issues we have been discussing
>
> 1/ Should there be JUnit tests in the samples.
> 2/ Should the mvn poms distributed with the binary distribution run
the
> sample client used by the Ant build instead of/as well as the JUnit
test
> 3/ If you look at the binding-echo test (and the other extension
example
> tests) you see I put the clients used by ant in the test directory.
This
is
> inconsistent with the rest of the samples but consistent with the
desire
to
> demonstrate here how to build an extension, rather than how to build a
> client.
>
> My 2c on these
> 1/ I don't mind there being JUnit tests in the samples. I would bow to
> consensus if it's generally thought that this complicates the first
use
> experience but as a java developer I know what a junit test is and
it's
> quite nice to have a test on which to base new tests that I may use
with
> any
> changes I make
>
Having test cases in the samples does not complicate them IMO if they
are in separate src/test directories. It also helps application
developers understand how to test an SCA application.
If it is desirable for the samples to provide a Junit-style template,
I could easily restructure the client code to follow the JUnit
pattern of
having setUp(), testXXX() and tearDown() methods, invoked in turn by the
main() method.
I'm not sure I understand this. In a normal application, main() would
not call Junit test code. Junit tests would be on the side, sometimes
duplicating a little bit what the application is doing, for test
purposes with assertions to test expected results. I don't think that
having the application main() reuse test code is common practice, and it
will probably not work as test code is usually not packaged with the
application anyway.
I think the most important thing to decide is whether it's useful for
samples (as opposed to tests) to use JUnit directly or follow a
JUnit-like
structure. I think we should go with the consensus on this. I don't
think
we should clutter the samples with JUnit tests and non-JUnit clients
that
both contain almost identical code.
Let's think about how an application developer will develop the same
kind of application. He will write a main method(). He will write
separate test cases with more assertions around the business logic. He
may have to duplicate 6 or 7 lines of code between the main() and the
junit test but it's not necessarily bad.
Having the main() method call the test code is very confusing IMO and
not common practice.
Reorganizing the sample into separate methods, maybe more classes, just
to reuse 6 or 7 lines will probably look more complicated in the end.
If we move the tests out of the samples then they'll miss one important
part of development, testing...
As I come from the second point of view I support having tests in the
samples. I can imagine that the clients and the JUnit tests would
diverge to
alow for more imaginative sample outout than we have now.
+1 Yes, for example prints in the main, assertions in the test.
2/ I think people running ant and mvn expect different things. I.e.
having
> mvn run the junit tests run is not unexpected to a maven user. We may
> choose
> to have the poms here run the ant client as well for completeness,
e.gwe
> could engage your wrappers to do this.
>
This is what the sca/itest/samples poms are doing now in the code I
posted.
I don't think this should becaome part of the sca/samples poms
because it
pollutes the samples even more with artifacts that are there for test
purposes.
> 3/ This was the basis of my original question. The tests are not
shipped
in
> the sample jars (I still need to find out why) to the effect is
that the
> client is missed from the sample jar and the three extension example
tests
> don't run out of the box. Due to the clients location I constructed
the
ant
> scripts to build the test directory but had to filter out the unit
tests
> because JUnit does not ship. I would rather solve these problems by
moving
> the client code into main/ than removing the JUnit tests altogether.
>
Not including the test code in the target JARs is common practice, as
tests are run as part of the build process, but you obviously don't want
them in the finished prodiuct.
If one or two samples have put their client code in src/test, then they
need to fixed and this client code should move to src/main. If there is
no client with a main() but only a test case, then the sample is
incomplete and a client with a main() should be added, but I don't think
that we should change the whole packaging strategy and try to twist
Maven's arm to include test code in the JARs just to make these
incorrect samples work. IMO we should try to fix the samples first.
It seems strange to me that we are including the jar files in the distro
but not the compiled standalone class files from src/test. The sample
pom.xml files solve this problem for maven users by forcing a recompile
of src/main and src/test before running the sample. (This makes it
pretty much pointless including the jars as part of the distributed
samples, as they always get rebuilt in this step). This doesn't require
the src/test classes to be part of the jar file. We could do the
same in
build.xml for ant builds.
We can just flick the ant build over to make the run target depend on the
compile target to get the effect you have described.
I took a look at the code. Some comments:
>
> - I like the idea of the wrappers but I would maybe use them as
well as
the
> junit tests rather than instead of.
>
It's a nuisance to maintain two sets of almost identical code, and
rather
confusing for the user if we deliver both in the samples. I think it's
better to pick one flavour.
I would like to see the client and test code diverge to make the clients
more meaningful from a sample point of view.
Yes, me too. I'm not surprised to see test code follow the same steps
and exercise the same methods as the main application code, with a few
more checkings and assertions.
This is what application developers will do in their applications
anyway. They will have a main() and separate test cases, containing
variations of similar code for different purposes.
- The effect of removing the JUnit tests from the samples is that the
> samples don't get tested anymore. Well not to the extent that the
> assertions
> that are currently included in the JUnit tests are exercised
Good point. For tests, this is important. For samples, displaying the
result to the console is better than running a silent assertion. It's
things like this that make me think we should make a distinction
between tests and samples. The wrapper approach does run all the
samples as part of the build, but doesn't currently do any validation
of the results.
> - The itests aren't shipped at the moment in the binary release
AFAIK so
> the
> maven based samples wouldn't be available. We could fix this of
course
but
> would need discussion at this stage.
>
Tests are usually not shipped with binaries. JUnit tests are run as part
of a build. If you build, you start from the source again.
By maven-based samples I presume you mean samples that run when you
type mvn. With the code I checked in, mvn builds the samples but
doesn't run them. I'm confident that I could come up with a small pom
change that would run them as well (not within Junit), if that is
desired.
There's a lot of discussion here. For clarity, here's what I think we
should do.
1. Have the ant and mvn sample builds do the same thing or close to
the same thing.
Think that sounds like a good idea but I include tests in that.
+1 and +1 to include the tests in that.
2. Have both of these run a non-junit sample client, after compiling
any necessary files first.
see 1
All client samples should have a non-Junit sample client.
ant run -> run that client
ant test -> run the Junit tests
mvn -> does not run anything
mvn test -> run the Junit tests
3. Divide sample code between src/main and src/test on the basis of what
makes most sense for the reader (remembering that these are
samples).
For samples that build a jar that will be used as a runtime
extension,
putting the test code for this extension into src/test and keeping
it out of the jar file seems best. For "standalone" samples that
consist of 100% test code, it seems best to avoid splitting this
code
into src/main and src/test categories (pick either one). I don't
think there's any need to build a jar in these cases.
I would prefer to keep the test cases in src/test in all cases, as it's
common practice and will make more sense to the reader.
I believe that the configuration we have now embodies this thinking
4. Don't ship pre-built jars for the samples. With mvn and ant scripts
provided for all the samples, it's really easy for anyone to build
the necessary class and jar files themselves.
Already responded to this one in a separate thread. I would prefer to
keep to pre-build samples.
I'm happy to ditch them and there is a very tiny change to the ant
build if
they are not there.
5. Have the itest/samples module do automated testing of the samples
as part of the build, using Junit.
If we keep the test cases in the samples. What does that bring us?
Comments much appreciated.
Simon
--
Jean-Sebastien
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
