Comments inline.
Simon
Jean-Sebastien Delfino wrote:
Comments inline.
Simon Laws wrote:
I think there is a difference in the way that we are looking at these
samples.
- We could consider that the samples are purely to show SCA running in
various scenarios and nothing more, i.e. anything extra such as sample
tests
or extra build steps is superfluous.
- Alternatively we could consider the samples to be a demonstration
of, not
only the function of the SCA runtime, but of how you might start to build
software that uses SCA, for example, you will likely need some tests
and you
will want to be able to integrate the sample into your IDE of choice.
I prefer that approach. SCA is a programming model for application
developers. Samples show how you use develop an application. Test is an
important part of development :) So I think we should have Junit test
cases as part of the sample code.
I'm OK with this as long as they are distinct from the samples.
I'd think of these as "sample tests".
I have been thinking along the latter lines and stepping back I'm
thinking
maybe we need to do more on testing rather than less, i.e. include a
JUnit
target in the ant script. We also need some more IDE detail and I've
already
marked this as a TODO in the top level samples README.
An Ant test target would be nice to have, but not mandatory for this
release IMO. Even with an Ant target I think we should keep the current
structure, which I believe follows common development practice: tests in
a separate directory, and not packaged in target JARs or a binary distro.
Agreed.
Anyhow I think this is why we seem to be coming at this from different
angles. So probably need others opinions to see which way to go.
Some more comments in line taking this into account.
On 5/11/07, Simon Nash <[EMAIL PROTECTED]> wrote:
Thanks for the detailed review and comments. My responses are inline.
Simon
Simon Laws wrote:
> Hi, I was thinking about this problem the other way round
>
> At the moment the binary distribution builds with the following
structure
> (taking binding-echo and simple-bigbank as two examples)
>
> docs/
> lib/
> modules/
> samples/
> binding-echo
> src/
> main/
> extension sample code
> test/
> java/
> echo/
> EchoBindingClient.java
> EchoReferenceTestCase.java
> EchoServiceTestCase.java
> simple-bigbank/
> src/
> main/
> java/
> bigbank/
> client/
> BigBankClient.java
> test/
> java/
> bigbank/
> BigBankTestCase.java
>
> So there are a number of issues we have been discussing
>
> 1/ Should there be JUnit tests in the samples.
> 2/ Should the mvn poms distributed with the binary distribution run
the
> sample client used by the Ant build instead of/as well as the JUnit
test
> 3/ If you look at the binding-echo test (and the other extension
example
> tests) you see I put the clients used by ant in the test directory.
This
is
> inconsistent with the rest of the samples but consistent with the
desire
to
> demonstrate here how to build an extension, rather than how to build a
> client.
>
> My 2c on these
> 1/ I don't mind there being JUnit tests in the samples. I would bow to
> consensus if it's generally thought that this complicates the first
use
> experience but as a java developer I know what a junit test is and
it's
> quite nice to have a test on which to base new tests that I may use
with
> any
> changes I make
>
Having test cases in the samples does not complicate them IMO if they
are in separate src/test directories. It also helps application
developers understand how to test an SCA application.
I agree with keeping the tests separate from the client code.
If it is desirable for the samples to provide a Junit-style template,
I could easily restructure the client code to follow the JUnit
pattern of
having setUp(), testXXX() and tearDown() methods, invoked in turn by the
main() method.
I'm not sure I understand this. In a normal application, main() would
not call Junit test code. Junit tests would be on the side, sometimes
duplicating a little bit what the application is doing, for test
purposes with assertions to test expected results. I don't think that
having the application main() reuse test code is common practice, and it
will probably not work as test code is usually not packaged with the
application anyway.
This suggestion was in response to SimonL's point that people will
appreciate having a template that can be used to create their own
JUnit tests. I was suggesting a way that we could ship one piece of
client code that can run without JUnit but would have a JUnit-like
structure to also serve as that template. We are now converging
around a different approach of keeping clients and tests completely
separate, and in that environment this suggestion would not apply.
I think the most important thing to decide is whether it's useful for
samples (as opposed to tests) to use JUnit directly or follow a
JUnit-like
structure. I think we should go with the consensus on this. I don't
think
we should clutter the samples with JUnit tests and non-JUnit clients
that
both contain almost identical code.
Let's think about how an application developer will develop the same
kind of application. He will write a main method(). He will write
separate test cases with more assertions around the business logic. He
may have to duplicate 6 or 7 lines of code between the main() and the
junit test but it's not necessarily bad.
I think it depends how much is duplicated. In some of the samples,
the client-side logic is more or less identical between clients and
tests (and I'm not just talking about the "boilerplate" code at the
start and end). If we can improve these samples to have more realistic
client code and test code, in which the test code would not be virtually
the same as the client code, then I am fine with some minor amount of
duplication betwween these.
Having the main() method call the test code is very confusing IMO and
not common practice.
Reorganizing the sample into separate methods, maybe more classes, just
to reuse 6 or 7 lines will probably look more complicated in the end.
If we move the tests out of the samples then they'll miss one important
part of development, testing...
See my comments above on this.
As I come from the second point of view I support having tests in the
samples. I can imagine that the clients and the JUnit tests would
diverge to
alow for more imaginative sample outout than we have now.
+1 Yes, for example prints in the main, assertions in the test.
Agreed. I also think tests should exercise more cases and conditions,
whereas sample client code should focus more on clarity of exposition,
showing how to achieve a practical piece of functionality.
2/ I think people running ant and mvn expect different things. I.e.
having
> mvn run the junit tests run is not unexpected to a maven user. We may
> choose
> to have the poms here run the ant client as well for completeness,
e.gwe
> could engage your wrappers to do this.
>
This is what the sca/itest/samples poms are doing now in the code I
posted.
I don't think this should becaome part of the sca/samples poms
because it
pollutes the samples even more with artifacts that are there for test
purposes.
> 3/ This was the basis of my original question. The tests are not
shipped
in
> the sample jars (I still need to find out why) to the effect is
that the
> client is missed from the sample jar and the three extension example
tests
> don't run out of the box. Due to the clients location I constructed
the
ant
> scripts to build the test directory but had to filter out the unit
tests
> because JUnit does not ship. I would rather solve these problems by
moving
> the client code into main/ than removing the JUnit tests altogether.
>
Not including the test code in the target JARs is common practice, as
tests are run as part of the build process, but you obviously don't want
them in the finished prodiuct.
If one or two samples have put their client code in src/test, then they
need to fixed and this client code should move to src/main. If there is
no client with a main() but only a test case, then the sample is
incomplete and a client with a main() should be added, but I don't think
that we should change the whole packaging strategy and try to twist
Maven's arm to include test code in the JARs just to make these
incorrect samples work. IMO we should try to fix the samples first.
Agreed.
It seems strange to me that we are including the jar files in the distro
but not the compiled standalone class files from src/test. The sample
pom.xml files solve this problem for maven users by forcing a recompile
of src/main and src/test before running the sample. (This makes it
pretty much pointless including the jars as part of the distributed
samples, as they always get rebuilt in this step). This doesn't require
the src/test classes to be part of the jar file. We could do the
same in
build.xml for ant builds.
We can just flick the ant build over to make the run target depend on the
compile target to get the effect you have described.
It takes a bit more than that as the "compile" target currently
doesn't compile the test classes.
I took a look at the code. Some comments:
>
> - I like the idea of the wrappers but I would maybe use them as
well as
the
> junit tests rather than instead of.
>
It's a nuisance to maintain two sets of almost identical code, and
rather
confusing for the user if we deliver both in the samples. I think it's
better to pick one flavour.
I would like to see the client and test code diverge to make the clients
more meaningful from a sample point of view.
Yes, me too. I'm not surprised to see test code follow the same steps
and exercise the same methods as the main application code, with a few
more checkings and assertions.
This is what application developers will do in their applications
anyway. They will have a main() and separate test cases, containing
variations of similar code for different purposes.
See my comments above on this. The devil is in the details here.
I don't think the code should be almost identical between sample
clients and JUnit tests except for assertions and displaying output.
There will be some similar patterns, but tests should exercise many
more variations on the same theme (e.g., different argument values)
to cover different potential failure scenarios, whereas sample code
should focus more on showing best practice for common use cases and
providing a template for what application writers will want to do
in their applications.
- The effect of removing the JUnit tests from the samples is that the
> samples don't get tested anymore. Well not to the extent that the
> assertions
> that are currently included in the JUnit tests are exercised
Good point. For tests, this is important. For samples, displaying the
result to the console is better than running a silent assertion. It's
things like this that make me think we should make a distinction
between tests and samples. The wrapper approach does run all the
samples as part of the build, but doesn't currently do any validation
of the results.
> - The itests aren't shipped at the moment in the binary release
AFAIK so
> the
> maven based samples wouldn't be available. We could fix this of
course
but
> would need discussion at this stage.
>
Tests are usually not shipped with binaries. JUnit tests are run as part
of a build. If you build, you start from the source again.
Agreed.
By maven-based samples I presume you mean samples that run when you
type mvn. With the code I checked in, mvn builds the samples but
doesn't run them. I'm confident that I could come up with a small pom
change that would run them as well (not within Junit), if that is
desired.
There's a lot of discussion here. For clarity, here's what I think we
should do.
1. Have the ant and mvn sample builds do the same thing or close to
the same thing.
Think that sounds like a good idea but I include tests in that.
+1 and +1 to include the tests in that.
I'm OK with including the tests, if we separate the tests and client code
as discussed above.
2. Have both of these run a non-junit sample client, after compiling
any necessary files first.
see 1
All client samples should have a non-Junit sample client.
ant run -> run that client
ant test -> run the Junit tests
mvn -> does not run anything
mvn test -> run the Junit tests
Agreed. I presume that "ant" would just build and not run, so that
it is the same as "mvn" (see above discussion about ant and mvn doing
the same). The current pom.xml and build.xml will need to be changed
to make things work this way.
3. Divide sample code between src/main and src/test on the basis of what
makes most sense for the reader (remembering that these are
samples).
For samples that build a jar that will be used as a runtime
extension,
putting the test code for this extension into src/test and keeping
it out of the jar file seems best. For "standalone" samples that
consist of 100% test code, it seems best to avoid splitting this
code
into src/main and src/test categories (pick either one). I don't
think there's any need to build a jar in these cases.
I would prefer to keep the test cases in src/test in all cases, as it's
common practice and will make more sense to the reader.
Agreed.
I believe that the configuration we have now embodies this thinking
4. Don't ship pre-built jars for the samples. With mvn and ant scripts
provided for all the samples, it's really easy for anyone to build
the necessary class and jar files themselves.
Already responded to this one in a separate thread. I would prefer to
keep to pre-build samples.
See my other post on this on a separate thread.
I'm happy to ditch them and there is a very tiny change to the ant
build if
they are not there.
5. Have the itest/samples module do automated testing of the samples
as part of the build, using Junit.
If we keep the test cases in the samples. What does that bring us?
It exercises the non-JUnit sample client code to make sure it runs.
Simon
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
