RE: Testing strategies
A strategy I've seen that's worked great is a mix between interfaces and private implementations. This would require an associated Factory to look up the actual implementation. This gives you the choice of stubbing out the interface if it's needed, or creating mock objects for the different private implementations. Maybe this would be a choice for when the project becomes more stable on the external inteface side? Kyrre --- Jerome Louvel [EMAIL PROTECTED] wrote: Agreed, I've entered a design task for 2.0 M1 to remember this point: http://restlet.tigris.org/issues/show_bug.cgi?id=384 Best regards, Jerome -Message d'origine- De : [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] De la part de Tim Peierls Envoyé : mardi 20 novembre 2007 20:33 À : discuss@restlet.tigris.org Objet : Re: Testing strategies On Nov 20, 2007 1:55 PM, Sean Landis [EMAIL PROTECTED] wrote: Stated differently, maybe when the convergence is over, it will be much safer to move to interfaces. The nearly immutable nature of interfaces won't be such an issue since the API questions will have mostly been answered. And presumably a compatibility layer could be maintained, at least through 2.x, to help 1.x clients migrate at their own speed. --tim Kyrre Kristiansen ___ Yahoo! Answers - Got a question? Someone out there knows the answer. Try it now. http://uk.answers.yahoo.com/
Re: Testing strategies
On Nov 20, 2007 9:06 AM, Jerome Louvel [EMAIL PROTECTED] wrote: Interfaces are useful in published APIs when they contain a very small and stable set of methods that you know won't need to expand. Agreed. A good example is the java.lang.Comparable interface. A counter example is the List interface. Imagine that Sun adds new methods to it, how many lines of code would need to be changed to accommodate the evolution? How many Applets/Servlets would break? That's not a counter-example, it's another example. No one should be thinking about changing List, any more than they should be thinking about changing Comparable -- too much code depends on it being stable. The advantages of being able to pass around Lists and Maps instead of Vectors and Hashtables have been proven. The fact that no one can ever change the List interface makes it *more* appropriate for use in public APIs. Other reasons to prefer classes over interfaces: 1) It makes the deprecation of methods easier because you can make the new method call the old logic (or the other way around), facilitating the progressive migration to the new method (like the core Java API do). OK, but this is just a corollary of the fact that interfaces are forever. If you really need to be constantly modifying a published API and yet retain complete backward compatibility with earlier versions, interfaces do indeed cause problems. (But should mutability of an API be the primary design concern?) 2) It reduces the conceptual weight of the API, no factories, no base/abstract classes to provide/understand. I disagree. A well-designed API involving interfaces is usually *easier* to understand *because* you don't have to learn about lots of types and implementation details. A good dependency injection framework can virtually eliminate the need for factories. 3) It allows the sharing of 'behavior' instead of just method signatures/contract with subclasses. This could just as easily go in the cons section. Subclasses are *forced* to share behavior with the parent class (or override it), which increases coupling. Here are the main cons I see: 1) Can't extend multiple parent classes 2) Not easily mockable 3) It is very difficult to design classes for extension, especially thread-safe classes. I'm not trying to talk you out of the current Restlet design, but I would rather have it described as the result of some tough design decisions on trade-offs concerning evolvability and implementation inheritance, not as classes are better than interfaces in APIs. --tim
RE: Testing strategies
Hi Tim, That's not a counter-example, it's another example. No one should be thinking about changing List, any more than they should be thinking about changing Comparable -- too much code depends on it being stable. The advantages of being able to pass around Lists and Maps instead of Vectors and Hashtables have been proven. The fact that no one can ever change the List interface makes it *more* appropriate for use in public APIs. There is indeed a lot of value in the List artifact. But it could very well have been provided as an abstract List class. I do see your point about usage of interfaces to be explicit about the fact that no evolution is possible in the future. 1) It makes the deprecation of methods easier... OK, but this is just a corollary of the fact that interfaces are forever. If you really need to be constantly modifying a published API and yet retain complete backward compatibility with earlier versions, interfaces do indeed cause problems. (But should mutability of an API be the primary design concern?) For an API such as the Collections API, it might be possible to fully embrace the features scope in one version, but for the Restlet case it didn't seem reasonable despite the fact that we had a lot of initial tests and feed-back before releasing v1. 2) It reduces the conceptual weight of the API... I disagree. A well-designed API involving interfaces is usually *easier* to understand *because* you don't have to learn about lots of types and implementation details. A good dependency injection framework can virtually eliminate the need for factories. Even if the public contract is a simple interface, don't you still need to understand the implementation details of the injected implementers ? 3) It allows the sharing of 'behavior'... This could just as easily go in the cons section. Subclasses are *forced* to share behavior with the parent class (or override it), which increases coupling. In the Restlet API, most classes are quite light in term of behavior. The hard logic is actually externalized in an engine via a SPI/'helper' mechanism (see org.restlet.util.Engine and org.restlet.util.Helper). 3) It is very difficult to design classes for extension, especially thread-safe classes. I'm not trying to talk you out of the current Restlet design, but I would rather have it described as the result of some tough design decisions on trade-offs concerning evolvability and implementation inheritance, not as classes are better than interfaces in APIs. This is a better description of the subject indeed, and one that reflects the design process that took place to design the v1 of the Restlet API. For example, the initial versions had interfaces for all the artifacts. Feed-back, discussions, readings led me to reconsider this approach. I'm open to try a different path for the design of the v2 of the API. At this point, we will have more experience on our problem domain and we will be able to afford more radical changes. Best regards, Jerome
Re: Testing strategies
I'm not trying to talk you out of the current Restlet design, but I would rather have it described as the result of some tough design decisions on trade-offs concerning evolvability and implementation inheritance, not as classes are better than interfaces in APIs. This is a better description of the subject indeed, and one that reflects the design process that took place to design the v1 of the Restlet API. For example, the initial versions had interfaces for all the artifacts. Feed-back, discussions, readings led me to reconsider this approach. I'm open to try a different path for the design of the v2 of the API. At this point, we will have more experience on our problem domain and we will be able to afford more radical changes. I had pushed a little bit against the move away from interfaces. I didn't like it but I understood that a design goal of Restlet was to converge onto an API through experience rather than try to get it right up front. Given that poorly charted territory was being navigated, I felt this was a helpful approach. I am glad to hear Jerome, that you are willing to reconsider the decision about interfaces. Stated differently, maybe when the convergence is over, it will be much safer to move to interfaces. The nearly immutable nature of interfaces won't be such an issue since the API questions will have mostly been answered. Sean
Re: Testing strategies
On Nov 20, 2007 1:55 PM, Sean Landis [EMAIL PROTECTED] wrote: Stated differently, maybe when the convergence is over, it will be much safer to move to interfaces. The nearly immutable nature of interfaces won't be such an issue since the API questions will have mostly been answered. And presumably a compatibility layer could be maintained, at least through 2.x, to help 1.x clients migrate at their own speed. --tim
Re: Testing strategies
Hi Justin, The usage of interfaces is discouraged in public APIs because they prevent changes in later releases (breaking existing implementation classes). I'm not familiar with EasyMock, but isn't it possible to generate mock objects that are subclasses of other classes, like a MockRequest extending a Request and setting custom values ? Could you describe a typical test that you would like to write, and we can discuss how we could write it ? Best regards, Jerome 2007/11/18, Justin Makeig [EMAIL PROTECTED]: I'm curious if and how others are writing unit tests for Restlet applications. In previous J2EE web apps I've written I've made heavy use of Easy Mock http://www.easymock.org/ for mocking collaborators and replaying scripts to validate behavior. Mocks lend themselves very well to interface-centric architectures, such as the style that Spring encourages. However, types, such as Context, Request, and Response are all concrete classes, requiring runtime contortions to mock or stub implementations, for example to set the attributes on a Request. The tests in the trunk don't seem very illustrative. Any suggestions would be much appreciated. Thanks. Justin
Re: Testing strategies
Jerome, I know you've made this statement before (interfaces in public APIs), and since it is your project, I've not pressed you about it, but I'd still like to see some evidence that this is more than just personal opinion. As for unit testing Restlets, I've found them much easier to test than servlets, primarily because there's no need to mock request objects - you can just create and initialize a Request and feed it directly to your resource via getRepresentation() (or put(), post(), etc). --Chuck On Nov 19, 2007 4:39 AM, Jerome Louvel [EMAIL PROTECTED] wrote: Hi Justin, The usage of interfaces is discouraged in public APIs because they prevent changes in later releases (breaking existing implementation classes). I'm not familiar with EasyMock, but isn't it possible to generate mock objects that are subclasses of other classes, like a MockRequest extending a Request and setting custom values ? Could you describe a typical test that you would like to write, and we can discuss how we could write it ? Best regards, Jerome 2007/11/18, Justin Makeig [EMAIL PROTECTED]: I'm curious if and how others are writing unit tests for Restlet applications. In previous J2EE web apps I've written I've made heavy use of Easy Mock http://www.easymock.org/ for mocking collaborators and replaying scripts to validate behavior. Mocks lend themselves very well to interface-centric architectures, such as the style that Spring encourages. However, types, such as Context, Request, and Response are all concrete classes, requiring runtime contortions to mock or stub implementations, for example to set the attributes on a Request. The tests in the trunk don't seem very illustrative. Any suggestions would be much appreciated. Thanks. Justin
Re: Testing strategies
On 11/18/07, Justin Makeig [EMAIL PROTECTED] wrote:
I'm curious if and how others are writing unit tests for Restlet
applications. In previous J2EE web apps I've written I've made heavy
use of Easy Mock http://www.easymock.org/ for mocking collaborators
and replaying scripts to validate behavior. Mocks lend themselves
very well to interface-centric architectures, such as the style that
Spring encourages. However, types, such as Context, Request, and
Response are all concrete classes, requiring runtime contortions to
mock or stub implementations, for example to set the attributes on a
Request. The tests in the trunk don't seem very illustrative. Any
suggestions would be much appreciated. Thanks.
We did the junit tests quite life-like in this fashion:
public abstract class ClientTest extends TestCommon {
public static final int PORT = 8977;
public static final String ROOT_URL = http://localhost:; + PORT +
/;
public static final String BASE_URL = ROOT_URL + v1/;
private static RestApplication server;
@BeforeClass
public synchronized static void startServer() throws Exception {
stopServer();
server = new RestApplication();
server.startServer(PORT);
}
@AfterClass
public synchronized static void stopServer() throws Exception {
if (server != null) {
server.stopServer();
}
}
@BeforeClass
public synchronized static void registerAdminUser() {
username = null;
useruri = null;
DAOFactory daoFactory = DAOFactory.getFactory();
user = new User();
password = User.generatePassword();
user.setPassword(password);
user.setAdmin(true);
daoFactory.getUserDAO().create(user);
daoFactory.commit();
username = user.getUsername();
useruri = BASE_URL + users/ + username;
System.out.println(Registered + username + + password);
}
/**
* Make an authenticated GET request.
*
* @return
*/
public Request makeAuthRequest() {
Request request = new Request();
ChallengeResponse challengeResponse =
new ChallengeResponse(ChallengeScheme.HTTP_BASIC,
username, password);
request.setChallengeResponse(challengeResponse);
request.setMethod(Method.GET);
return request;
}
}
Then a simple test can be like this:
public class TestDataDoc extends ClientTest {
@Test
public void getAllXML() throws IOException, ParseException,
XmlException {
Request request = makeAuthRequest();
Client client = new Client(Protocol.HTTP);
request.setResourceRef(useruri + /data);
request.setMethod(Method.GET);
request.getClientInfo().getAcceptedMediaTypes().add(
new PreferenceMediaType(restType));
Response response = client.handle(request);
assertTrue(response.getStatus().isSuccess());
DatasDocument doc =
DatasDocument.Factory.parse(response.getEntity().getStream());
for (Data data : doc.getDatas().getDataArray()) {
assertTrue(data.getHref().startsWith(BASE_URL +
data));
}
}
@Test
public void create() throws IOException {
Request request = makeAuthRequest();
Client client = new Client(Protocol.HTTP);
request.setResourceRef(useruri + /data);
request.setMethod(Method.POST);
request.setEntity(datadoc, baclavaType);
Response response = client.handle(request);
assertTrue(response.getStatus().isSuccess());
justCreated = response.getRedirectRef();
}
@Test
public void createEmpty() throws IOException {
Request request = makeAuthRequest();
Client client = new Client(Protocol.HTTP);
request.setResourceRef(useruri + /data);
request.setMethod(Method.POST);
request.setEntity(, baclavaType);
Response response = client.handle(request);
assertEquals(Status.CLIENT_ERROR_BAD_REQUEST,
response.getStatus());
assertNull(response.getRedirectRef());
}
}
--
Stian Soiland, myGrid team
School of Computer Science
The University of Manchester
http://www.cs.man.ac.uk/~ssoiland/
Re: Testing strategies
Chuck, thanks for the input. In general, yes, Restlets are much simpler in their dependencies than servlets and thus easier to test. However, my struggle is extra cost of writing stubs vs. mocks http:// martinfowler.com/articles/mocksArentStubs.html. Mocks allow one to just implement a test version of only the functionality you need as a collaborator to the (concrete) class being tested, rather than to subclass a stubbed out anemic version of a collaborator interface (i.e. its public methods). Additionally, something like Easy Mock allows you to replay a test script to ensure that the collaborators were used as expected. Justin On Nov 19, 2007, at 7:17 AM, Chuck Hinson wrote: Jerome, I know you've made this statement before (interfaces in public APIs), and since it is your project, I've not pressed you about it, but I'd still like to see some evidence that this is more than just personal opinion. As for unit testing Restlets, I've found them much easier to test than servlets, primarily because there's no need to mock request objects - you can just create and initialize a Request and feed it directly to your resource via getRepresentation() (or put(), post(), etc). --Chuck On Nov 19, 2007 4:39 AM, Jerome Louvel [EMAIL PROTECTED] wrote: Hi Justin, The usage of interfaces is discouraged in public APIs because they prevent changes in later releases (breaking existing implementation classes). I'm not familiar with EasyMock, but isn't it possible to generate mock objects that are subclasses of other classes, like a MockRequest extending a Request and setting custom values ? Could you describe a typical test that you would like to write, and we can discuss how we could write it ? Best regards, Jerome 2007/11/18, Justin Makeig [EMAIL PROTECTED]: I'm curious if and how others are writing unit tests for Restlet applications. In previous J2EE web apps I've written I've made heavy use of Easy Mock http://www.easymock.org/ for mocking collaborators and replaying scripts to validate behavior. Mocks lend themselves very well to interface-centric architectures, such as the style that Spring encourages. However, types, such as Context, Request, and Response are all concrete classes, requiring runtime contortions to mock or stub implementations, for example to set the attributes on a Request. The tests in the trunk don't seem very illustrative. Any suggestions would be much appreciated. Thanks. Justin
Re: Testing strategies
Stian,
I would consider what you've helpfully included below an integration,
not unit test. These are invaluable, but I'm trying to just test
small pieces of functionality in isolation one at a time, not the
whole stack. For example, if I were to test my Resource
implementation's put method, I only want to test its inner workings,
not all of the other stuff that it uses, such as my service layer or
actual Representations (presumably these will be tested elsewhere).
Maybe my approach is inappropriate. I'll do some more thinking and
coding and post my findings.
Thanks to everyone for the input.
Justin
On Nov 19, 2007, at 9:36 AM, Stian Soiland wrote:
On 11/18/07, Justin Makeig [EMAIL PROTECTED] wrote:
I'm curious if and how others are writing unit tests for Restlet
applications. In previous J2EE web apps I've written I've made heavy
use of Easy Mock http://www.easymock.org/ for mocking collaborators
and replaying scripts to validate behavior. Mocks lend themselves
very well to interface-centric architectures, such as the style that
Spring encourages. However, types, such as Context, Request, and
Response are all concrete classes, requiring runtime contortions to
mock or stub implementations, for example to set the attributes on a
Request. The tests in the trunk don't seem very illustrative. Any
suggestions would be much appreciated. Thanks.
We did the junit tests quite life-like in this fashion:
public abstract class ClientTest extends TestCommon {
public static final int PORT = 8977;
public static final String ROOT_URL = http://localhost:; + PORT
+ /;
public static final String BASE_URL = ROOT_URL + v1/;
private static RestApplication server;
@BeforeClass
public synchronized static void startServer() throws Exception {
stopServer();
server = new RestApplication();
server.startServer(PORT);
}
@AfterClass
public synchronized static void stopServer() throws Exception {
if (server != null) {
server.stopServer();
}
}
@BeforeClass
public synchronized static void registerAdminUser() {
username = null;
useruri = null;
DAOFactory daoFactory = DAOFactory.getFactory();
user = new User();
password = User.generatePassword();
user.setPassword(password);
user.setAdmin(true);
daoFactory.getUserDAO().create(user);
daoFactory.commit();
username = user.getUsername();
useruri = BASE_URL + users/ + username;
System.out.println(Registered + username + + password);
}
/**
* Make an authenticated GET request.
*
* @return
*/
public Request makeAuthRequest() {
Request request = new Request();
ChallengeResponse challengeResponse =
new ChallengeResponse(ChallengeScheme.HTTP_BASIC,
username, password);
request.setChallengeResponse(challengeResponse);
request.setMethod(Method.GET);
return request;
}
}
Then a simple test can be like this:
public class TestDataDoc extends ClientTest {
@Test
public void getAllXML() throws IOException, ParseException,
XmlException {
Request request = makeAuthRequest();
Client client = new Client(Protocol.HTTP);
request.setResourceRef(useruri + /data);
request.setMethod(Method.GET);
request.getClientInfo().getAcceptedMediaTypes().add(
new PreferenceMediaType(restType));
Response response = client.handle(request);
assertTrue(response.getStatus().isSuccess());
DatasDocument doc =
DatasDocument.Factory.parse(response.getEntity().getStream());
for (Data data : doc.getDatas().getDataArray()) {
assertTrue(data.getHref().startsWith(BASE_URL +
data));
}
}
@Test
public void create() throws IOException {
Request request = makeAuthRequest();
Client client = new Client(Protocol.HTTP);
request.setResourceRef(useruri + /data);
request.setMethod(Method.POST);
request.setEntity(datadoc, baclavaType);
Response response = client.handle(request);
assertTrue(response.getStatus().isSuccess());
justCreated = response.getRedirectRef();
}
@Test
public void createEmpty() throws IOException {
Request request = makeAuthRequest();
Client client = new Client(Protocol.HTTP);
Re: Testing strategies
On Nov 19, 2007 3:35 PM, Jerome Louvel [EMAIL PROTECTED] wrote: Hi Chuck, Jerome, I know you've made this statement before (interfaces in public APIs), and since it is your project, I've not pressed you about it, but I'd still like to see some evidence that this is more than just personal opinion. Initially, the API had many interfaces but some (Henry Story I think) pointed me that public APIs based on interfaces are fragile because once they are published they can't easily evolve. If you have interface A with method A1 in version 1.0, then users will implement it in their custom classes, this is fine, they can easily mock them. Then, for version 2.0, the need to add method A2 appears and is added to the A interface. When the API is published, users try to upgrade and realize that their custom classes don't implement method A2, or implement it with a different purpose or different arguments. Their code is broken and the API break the ascendant compatibility. [...] I see the point, but remain unconvinced; with modern refactoring and dependency management tools, I'm not sure that this is an actual problem. I'd also point out that many people view the addition of a method to a base class (abstract or otherwise) as a breaking change (because existing clients may have already implemented a method with the same signature in existing subclasses). If you change an API, it doesnt seem unreasonable to me to expect that implementors will have to change their implementation. To me, it seems like a case of throwing the baby out with the bath water. Versioning's hard, and there are appropriate and inappropriate uses of interfaces, but to make a blanket statement that interfaces shouldn't be used in a published API seems unwarranted. I was actually hoping for a pointer to a discussion about the pros and cons of using abstract classes instead of the interfaces in published APIs, but I haven't been able to find anything - which makes me wonder whether it's really that big a deal. --Chuck
