Hi Krum, btw, can you please
attach a test maven project that is used to build the CXF-based war to
https://issues.apache.org/jira/browse/CXF-4121 ?
I'll run my own tests against it too
Thanks, Sergey
On 23/02/12 15:51, Sergey Beryozkin wrote:
Hi,
On 23/02/12 14:50, Bakalsky, Krum wrote:
Hi Sergey,
Thanks for the experimenting you have performed.
[Yes, I checked the WAR archive, and it doesn't contain any Jersey
stuff inside.]
The reason I asked is that I recall Jersey Servlet scanning custom wars
for Applications on Glassfish, but indeed, should not be the case in the
Tomcat installation.
Which client/testing framework do you use in the pasted code snippet ?
I'm using CXF client API:
http://cxf.apache.org/docs/jax-rs-client-api.html.
it uses HttpUrlConnection... The client code runs in a diff process...
I am asking that, because strangely enough we got much better
performance when switching the client framework from Meterware's
HttpUnit to Apache HttpClient. Could that really bring the significant
difference ?
I'm not sure, given it is the basic processing on the client side,
Maybe the responses that CXF returns are not exactly the same as those
that Jersey returns, and are somehow processed much slower by the
HttpUnit framework (maybe we have some issue of lazy loading the
response body, headers ... or just the HTTP parser is more mature in
Apache HttpClient...).
FYI, here is tcp trace fragment of what CXF returns in case of negative
404:
HTTP/1.1 404 Not Found
Date: Thu, 23 Feb 2012 15:47:20 GMT
Content-Length: 0
Server: Jetty(6.1.15)
similarly for Tomcat
HTTP/1.1 404 Not Found
Server: Apache-Coyote/1.1
Date: Thu, 23 Feb 2012 15:49:07 GMT
Content-Length: 0
Thanks a lot once again for the serious attention that you are paying
on that topic!
not at all, I'm motivated to get CXF JAX-RS producing good perf results
:-),
thanks for the feedback,
Sergey
Kindest Regards,
Krum.
-----Original Message-----
From: Sergey Beryozkin [mailto:[email protected]]
Sent: Thursday, February 23, 2012 2:16 PM
To: [email protected]
Subject: Re: REST performance discrepancies between CXF and Jersey
Hi Krum
On 22/02/12 16:09, Bakalsky, Krum wrote:
Just a short update: the only way I could successfully enter my
custom exception mapper with the debugger, was to annotate it with
the @Provider annotation: I found that after debugging
this<http://grepcode.com/file/repo1.maven.org/maven2/org.apache.cxf/cxf-rt-frontend-jaxrs/2.5.1/org/apache/cxf/jaxrs/utils/ResourceUtils.java>
code (if
(c.getAnnotation<http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b14/java/lang/Class.java>(Provider<http://grepcode.com/file/repo1.maven.org/maven2/javax.ws.rs/jsr311-api/1.1.1/javax/ws/rs/ext/Provider.java>.class)
!= null) )
Still 404 is slow ... So seems to be not an exception mapper issue...
My tests show 0.006 - 0.007 for the negative case.
I've deployed the war to the jetty server, run the 100 iterations first
to warm things up - that gave me 0.009
from then on I get 0.006 - 0.007 all the time which I guess is still not
perfect but definitely not 0.5 :-)
In the server I just do
throw new WebApplicationException(404) from within the resource method,
on the client:
long total = 0;
for (int i = 0; i< 100; i++) {
long l1 = System.currentTimeMillis();
try {
client.useSearchService();
} catch (Exception ex) {
}
long l2 = System.currentTimeMillis();
total += l2-l1;
}
System.out.println("Totall in milliseconds: " + total / 100);
where
useSearchService() looks like this:
WebClient wc = WebClient.create("http://localhost:"; + port +
"/services/personservice/search");
wc.accept(MediaType.APPLICATION_XML);
wc.query("name", "Fred", "Lorraine");
wc.get(Response.class);
so nothing unusual there...
What I'm still curious about is why you got the unexpected improvements
after erroneously registering the custom mapper - there must be some
explanation for that :-). Can you also please check Jersey is not loaded
in the process which runs the CXF endpoint ?
btw, run the war in Tomcat too - 0.007 all the time
Thanks, Sergey
-----Original Message-----
From: Sergey Beryozkin [mailto:[email protected]]
Sent: Wednesday, February 22, 2012 5:17 PM
To: [email protected]
Subject: Re: REST performance discrepancies between CXF and Jersey
On 22/02/12 15:11, Bakalsky, Krum wrote:
Yes, that's right :)
So, maybe the recipe for being faster here is: just throw some
exception within getClasses() ;-) - just kidding...
LOL :-)
Maybe I should try to see with some breakpoints where exactly the
work flow goes through ...
I'm really intrigued now in seeing where the cost savings are coming
from in this case :-), please try to check if you can get a chance...
Anyway, do you know how to register custom exception mappers, in the
non-Spring case ? Is the getClasses() result the only way possible
for that ?
as far as providers are concerned, they can be returned in either
getClasses() - the runtime will have to load them or in getSingletons(),
alongside the root resource classes if any...
I'm finalizing my current main task soon, so will be looking into it all
seriously soon
Cheers, Sergey
-----Original Message-----
From: Sergey Beryozkin [mailto:[email protected]]
Sent: Wednesday, February 22, 2012 4:30 PM
To: [email protected]
Subject: Re: REST performance discrepancies between CXF and Jersey
Hi, By the way,
On 22/02/12 14:19, Sergey Beryozkin wrote:
Hi Krum,
Before commenting further, let me ask you how do you take the
measurements ? Do you check on the client side ? On the server side ?
Thanks, Sergey
On 22/02/12 13:49, Bakalsky, Krum wrote:
Hi Sergey, again,
I am afraid that we are in the middle of some big
misunderstanding, me
being the one who has started it. Unfortunately, it turned out that I
didn't quite realize, that javax.ws.rs.core.Application#getClasses()
returns a non-modifiable collection (Collections.emptySet()), and in
my exception mapper I used:
public Set<Class<?>> getClasses() {
// TODO Auto-generated method stub
Set<Class<?>> classes = super.getClasses();
classes.add(SimpleApplicationExceptionMapper.class);
return classes;
}
which probably leads to an java.lang.UnsupportedOperationException at
runtime, and thus to some work flow/behavior that I am not aware of.
This made mu numbers faster, but obviously the scenario is broken, so
the results are irrelevant.
This actually surprises me. Assuming CXFNonSpringJaxrsServlet ignored
getClasses() due to this exception, then it means that the default CXF
mapper should've been used, otherwise you'd not see 404 on the client
side, you'd just get 500 due to your custom exception simply escaping.
Yet apparently the results have become better :-) - thins is something
I'd like to understand...
Next, I took a look at some examples, like
this<http://grepcode.com/file/repo1.maven.org/maven2/com.sun.jersey.samples/jersey-ejb/1.11/com/sun/jersey/samples/jersey_ejb/resources/MyApplication.java?av=f>
one, and modified my code accordingly. Unfortunately, I got the same
results: 0.500 seconds for the 404 cases. I am not even sure that my
exception mapper was really used, since when I inserted some dumps,
and even when I threw some exceptions in my toResponse() method, I
didn't see them nowhere.
Next, I made a step further, and patched the
WebApplicationExceptionMapper that comes with CXF to look like this:
public class WebApplicationExceptionMapper implements
ExceptionMapper<WebApplicationException> {
public Response toResponse(WebApplicationException ex) {
return ex.getResponse();
}
}
I made sure that it was really used. Unfortunately, I got the same
results: roundtrip time was again ~0.500 seconds.
I am really not sure whether the CXF-4121 bug makes sense any more.
Problem is still here ;(
Sure, we'll get to the bottom of it, however, lets try to see where the
unexpected improvement after registering a custom mapper came from...
Actually, I think I may know why your custom mapper may be ignored. It
should have a @Provider annotation, for it to be recognized as a
provider in the list returned from getClasses()
Cheers, Sergey
Kindest Regards,
Krum.
-----Original Message-----
From: Bakalsky, Krum
Sent: Wednesday, February 22, 2012 11:35 AM
To: '[email protected]'
Subject: RE: REST performance discrepancies between CXF and Jersey
Hi Sergey,
I have just made a test run with the modified WebApplicationException
mapper that you have given a link to below, and it works fine: it
also
shows 10 times decrease of roundtrip duration as well. So, it seems
that you have really optimized it pretty good already.
Sure, I don't have any particular objections or concerns as to how to
approach the change that we are talking about. I'd guess that as long
as the application has a way to configure its own exception mapper
(which is already the case) so that 404 overhead is gone for it,
things are fine. I wouldn't know for those implications and
complications that adding a property could lead to. Thanks for
sharing.
We haven't identified particular 'slow' code fragments, other than
the
404 processing. If we do, we will sure let you know. (or if you want,
you could share your suspicions and give us some hint of where to
take
a look at ?)
Kindest Regards,
Krum.
-----Original Message-----
From: Sergey Beryozkin [mailto:[email protected]]
Sent: Tuesday, February 21, 2012 8:34 PM
To: [email protected]
Subject: Re: REST performance discrepancies between CXF and Jersey
Hi Krum,
On 20/02/12 16:47, Bakalsky, Krum wrote:
Hi,
Yes, thanks for the comments, they sound very logical to explaining
the numbers from all the variants we tested.
Didn't know that the default exception mapper was developed as a
result of a user demand as well.
For the record: how could I disable it ? I am registering the Simple
mapper within the returned set of the Application#getClasses()
method
call - so maybe I should remove it there, i.e. from the
super.getClasses() list, or it is in a different manner ?
Yes, I think that a servlet context parameter would be the most
flexible and elegant solution, for configuring the desired behavior
for non-200/exceptional cases.
I've spent some today on optimizing a couple of things,
see the updated default WebApplicationException mapper:
http://svn.apache.org/repos/asf/cxf/trunk/rt/frontend/jaxrs/src/main/java/org/apache/cxf/jaxrs/impl/WebApplicationExceptionMapper.java
I think it's more effective now, yes still it is configurable and
able
to interact with CXF FaultLoggers and optionally fine-log the
exception
message.
I was thinking yesterday of adding a property to get this default
mapper
ignored. I'm not sure now it's worth going this route.
One way to 'disable' it is to offer a custom WebApplicationException
mapper, something which we've explored. Another one - is to add this
would be property but I've realized that it's not that good for CXF
after all: this would require the CXF servlet to process unmapped
WebApplicationExceptions to be compliant and this would
effectively mean
repeating at the servlet level what is done at the CXF chain level
but
with limitations, ex, the Response produced by the mapper would
still be
routed through the out filters and indeed the optional CXF out faulty
interceptors which can be handy. Say we can have an out filter which
time-stamps all the responses - this would be ignored at the servlet
level and would effectively have to be duplicated if needed...
The other thing is that CXFNonSpringJaxrsServlet and servlet filters
will not be necessarily installed in some OSGI containers by default.
So it appears to be the best way is to optimize the default mapper
which
I hope I did, and it would be always possible to offer custom mappers
instead on per-Application basis.
Let me know please if you have any concerns with the above,
In meantime I also updated some code on the in chain to optimize
the way
the HTTP path to match is calculated - it may well save us few
milliseconds :-)
If you can share some more results from your tests which point to few
unoptimized code spots then it would be good; we are going to
review few
code paths anyway,
Thanks, Sergey
Kindest Regards and many thanks again,
Krum.
-----Original Message-----
From: Sergey Beryozkin [mailto:[email protected]]
Sent: Monday, February 20, 2012 5:49 PM
To: [email protected]
Subject: Re: REST performance discrepancies between CXF and Jersey
Hi,
On 20/02/12 15:28, Sergey Beryozkin wrote:
Hi Krum
On 20/02/12 15:05, Bakalsky, Krum wrote:
Hi Sergey,
Wow! I tested your idea and it produced dramatically better
results!
The roundtrip time for 404 requests decreased by a factor of 10
(from
0.500 seconds down to 0.030 seconds). That is a huge success
for our
investigation, and I believe that our observation is really
true that
the overhead is spent in the error handling code. Just out of
curiosity: with the same SimpleApplicationMapper:
Thanks for the confirmation, so we've identified the spot where the
problem is :-)
public class SimpleApplicationExceptionMapper implements
ExceptionMapper<WebApplicationException> {
@Override
public Response toResponse(WebApplicationException exception) {
return Response.status(404).build();
}
}
Jersey behaved very similarly to CXF in terms of roundtrip time.
However, without this class, Jersey performed even faster -
most of
the times were 10 times faster, i.e. around 0.003 seconds,
while in
the CXF case ... you already know it :) - 0.500 seconds
Forgot to comment to this one... I think the fact that both
frameworks
show approximately the same time with
SimpleApplicationExceptionMapper
in place and that Jersey scales really well without it is that
Jersey
basically either writes to HTTP out stream or propagates the
exception
up to the ServletException filter immediately if no custom mapper is
available.
In case of CXF the immediate propagation will also happen if no
custom
mapper is available. The reason the default WebApplicationException
Mapper was installed was to do with the fact that few users were not
quite happy with having to write their custom mappers in order to
report
differently to the exceptions generated at the runtime level or
some of
the providers' level or indeed having to write a filter catching the
escaped exceptions there...
What I might need to do additionally is to introduce a contextual
property that will tell the runtime to ignore even the default
WebApplicationException mapper, for example, when the user wishes to
get all the exceptions propagated to the filter level, to
simplify the
performance data comparison, etc
Cheers, Sergey
Thank a lot for your idea and suggestion! It definitely shed
light on
where the discrepancies come from.
I've opened
https://issues.apache.org/jira/browse/CXF-4121
I suspect now that it is the specific lines which get the basic
warning
message from the resource bundle are main 'culprits', I guess I
will
just have an internal final static String - will need to
experiment a
bit...It is good even the default providers can be
customized/replaced,
but I'll look into trying to make the default exception mapper
performing a bit better :-)
Thanks, Sergey
Kindest Regards,
Krum.
-----Original Message-----
From: Sergey Beryozkin [mailto:[email protected]]
Sent: Friday, February 17, 2012 6:14 PM
To: [email protected]
Subject: Re: REST performance discrepancies between CXF and Jersey
Hi Krum,
On 17/02/12 15:26, Bakalsky, Krum wrote:
Hi to all Apache CXF users!,
I would kindly like to bring to your attention a particular
problem
that we are currently being stuck into. We are experimenting
with a
simple web application, which exposes its functionality via
REST. We
are using JAX-RS/JSON, no SOAP, no JAX-WS. We deploy our
application
on top of Tomcat 7, and test both against CXF 2.5.2 and Jersey as
REST frameworks.
For calculating our performance measurements, we simply
execute REST
requests, and in the client we measure the time that the
roundtrip
takes. Apart from that, on the server side, we measure as well
the
time that our REST resource consumes, i.e. the method call
time. In
summary, we have some huge differences between CXF and Jersey
- in
favor of Jersey being much faster - in the cases where we test
against non-existing resources. In all other cases, the
numbers we
got were really very close for the two REST frameworks.
But when testing against non-existing REST paths, the
roundtrip time
in CXF case is approximately 0.5 seconds, while in the case of
Jersey
it is 0.01 seconds. We think that this is quite strange, and
probably
we are not using CXF in its proper configuration. Could you
please
give us some hints ? Maybe we have to tune and tweak it a
little bit,
so that we get the best out of it.
The server side time, in the case of CXF, is rather small - 0.001
seconds, so we wonder where is that half second spent in ? In our
application logic, in the case when the path is not correct,
we throw
javax.ws.rs.WebApplicationException and with a debugger we saw
that
this exception is getting caught in CXF, and heavily processed
further down the stack.
Are you aware of such kind of issues, with CXF handling the 404
situation in a very heavyweight manner ? Maybe it does some
thorough
processing to try to map the URI to some existing REST
resource, I
don't know ... maybe performance here could depend on the
particular
JSON framework that is being used. ...
I suspect it could be the default WebApplicationExceptionMapper
which is
to blame.
In CXF, a given Exception, once wrapped in a Response, is
treated like
any other regular response, but I'm not sure it would explain the
difference.
Can you give me a favor please and register a custom
WebApplicationException mapper which would only return:
Response.status(exception.getStatus()).build() ?
Have a look please at the default mapper's code:
http://svn.apache.org/repos/asf/cxf/trunk/rt/frontend/jaxrs/src/main/java/org/apache/cxf/jaxrs/impl/WebApplicationExceptionMapper.java
As you can see, we are trying to go to some length there to
figure out
what and how to report a given exception and I reckon it all adds
up to
the response time.
Please try eliminating the default mapper and let us know the
results
Thanks, Sergey
Kindest Regards,
Krum.
--
--
Sergey Beryozkin
Talend Community Coders
http://coders.talend.com/
Blog: http://sberyozkin.blogspot.com
