Re: REST performance discrepancies between CXF and Jersey

Wed, 22 Feb 2012 06:30:11 -0800 

Hi, By the way,

On 22/02/12 14:19, Sergey Beryozkin wrote:

Hi Krum,

Before commenting further, let me ask you how do you take the
measurements ? Do you check on the client side ? On the server side ?

Thanks, Sergey


On 22/02/12 13:49, Bakalsky, Krum wrote:

Hi Sergey, again,

I am afraid that we are in the middle of some big misunderstanding, me
being the one who has started it. Unfortunately, it turned out that I
didn't quite realize, that javax.ws.rs.core.Application#getClasses()
returns a non-modifiable collection (Collections.emptySet()), and in
my exception mapper I used:

public Set<Class<?>> getClasses() {
// TODO Auto-generated method stub
Set<Class<?>> classes = super.getClasses();
classes.add(SimpleApplicationExceptionMapper.class);
return classes;
}

which probably leads to an java.lang.UnsupportedOperationException at
runtime, and thus to some work flow/behavior that I am not aware of.
This made mu numbers faster, but obviously the scenario is broken, so
the results are irrelevant.
This actually surprises me. Assuming CXFNonSpringJaxrsServlet ignored getClasses() due to this exception, then it means that the default CXF mapper should've been used, otherwise you'd not see 404 on the client side, you'd just get 500 due to your custom exception simply escaping. Yet apparently the results have become better :-) - thins is something I'd like to understand... 




Next, I took a look at some examples, like
this<http://grepcode.com/file/repo1.maven.org/maven2/com.sun.jersey.samples/jersey-ejb/1.11/com/sun/jersey/samples/jersey_ejb/resources/MyApplication.java?av=f>
one, and modified my code accordingly. Unfortunately, I got the same
results: 0.500 seconds for the 404 cases. I am not even sure that my
exception mapper was really used, since when I inserted some dumps,
and even when I threw some exceptions in my toResponse() method, I
didn't see them nowhere.

Next, I made a step further, and patched the
WebApplicationExceptionMapper that comes with CXF to look like this:

public class WebApplicationExceptionMapper implements
ExceptionMapper<WebApplicationException> {

public Response toResponse(WebApplicationException ex) {
return ex.getResponse();
}
}

I made sure that it was really used. Unfortunately, I got the same
results: roundtrip time was again ~0.500 seconds.

I am really not sure whether the CXF-4121 bug makes sense any more.
Problem is still here ;(
Sure, we'll get to the bottom of it, however, lets try to see where the unexpected improvement after registering a custom mapper came from...
Actually, I think I may know why your custom mapper may be ignored. It should have a @Provider annotation, for it to be recognized as a provider in the list returned from getClasses() 

Cheers, Sergey



Kindest Regards,
Krum.

-----Original Message-----
From: Bakalsky, Krum
Sent: Wednesday, February 22, 2012 11:35 AM
To: '[email protected]'
Subject: RE: REST performance discrepancies between CXF and Jersey

Hi Sergey,

I have just made a test run with the modified WebApplicationException
mapper that you have given a link to below, and it works fine: it also
shows 10 times decrease of roundtrip duration as well. So, it seems
that you have really optimized it pretty good already.

Sure, I don't have any particular objections or concerns as to how to
approach the change that we are talking about. I'd guess that as long
as the application has a way to configure its own exception mapper
(which is already the case) so that 404 overhead is gone for it,
things are fine. I wouldn't know for those implications and
complications that adding a property could lead to. Thanks for sharing.

We haven't identified particular 'slow' code fragments, other than the
404 processing. If we do, we will sure let you know. (or if you want,
you could share your suspicions and give us some hint of where to take
a look at ?)

Kindest Regards,
Krum.

-----Original Message-----
From: Sergey Beryozkin [mailto:[email protected]]
Sent: Tuesday, February 21, 2012 8:34 PM
To: [email protected]
Subject: Re: REST performance discrepancies between CXF and Jersey

Hi Krum,
On 20/02/12 16:47, Bakalsky, Krum wrote:

Hi,

Yes, thanks for the comments, they sound very logical to explaining
the numbers from all the variants we tested.
Didn't know that the default exception mapper was developed as a
result of a user demand as well.

For the record: how could I disable it ? I am registering the Simple
mapper within the returned set of the Application#getClasses() method
call - so maybe I should remove it there, i.e. from the
super.getClasses() list, or it is in a different manner ?

Yes, I think that a servlet context parameter would be the most
flexible and elegant solution, for configuring the desired behavior
for non-200/exceptional cases.


I've spent some today on optimizing a couple of things,
see the updated default WebApplicationException mapper:

http://svn.apache.org/repos/asf/cxf/trunk/rt/frontend/jaxrs/src/main/java/org/apache/cxf/jaxrs/impl/WebApplicationExceptionMapper.java


I think it's more effective now, yes still it is configurable and able
to interact with CXF FaultLoggers and optionally fine-log the exception
message.

I was thinking yesterday of adding a property to get this default mapper
ignored. I'm not sure now it's worth going this route.

One way to 'disable' it is to offer a custom WebApplicationException
mapper, something which we've explored. Another one - is to add this
would be property but I've realized that it's not that good for CXF
after all: this would require the CXF servlet to process unmapped
WebApplicationExceptions to be compliant and this would effectively mean
repeating at the servlet level what is done at the CXF chain level but
with limitations, ex, the Response produced by the mapper would still be
routed through the out filters and indeed the optional CXF out faulty
interceptors which can be handy. Say we can have an out filter which
time-stamps all the responses - this would be ignored at the servlet
level and would effectively have to be duplicated if needed...

The other thing is that CXFNonSpringJaxrsServlet and servlet filters
will not be necessarily installed in some OSGI containers by default.

So it appears to be the best way is to optimize the default mapper which
I hope I did, and it would be always possible to offer custom mappers
instead on per-Application basis.

Let me know please if you have any concerns with the above,

In meantime I also updated some code on the in chain to optimize the way
the HTTP path to match is calculated - it may well save us few
milliseconds :-)

If you can share some more results from your tests which point to few
unoptimized code spots then it would be good; we are going to review few
code paths anyway,

Thanks, Sergey



Kindest Regards and many thanks again,
Krum.

-----Original Message-----
From: Sergey Beryozkin [mailto:[email protected]]
Sent: Monday, February 20, 2012 5:49 PM
To: [email protected]
Subject: Re: REST performance discrepancies between CXF and Jersey

Hi,
On 20/02/12 15:28, Sergey Beryozkin wrote:

Hi Krum
On 20/02/12 15:05, Bakalsky, Krum wrote:

Hi Sergey,

Wow! I tested your idea and it produced dramatically better results!
The roundtrip time for 404 requests decreased by a factor of 10 (from
0.500 seconds down to 0.030 seconds). That is a huge success for our
investigation, and I believe that our observation is really true that
the overhead is spent in the error handling code. Just out of
curiosity: with the same SimpleApplicationMapper:


Thanks for the confirmation, so we've identified the spot where the
problem is :-)


public class SimpleApplicationExceptionMapper implements
ExceptionMapper<WebApplicationException> {

@Override
public Response toResponse(WebApplicationException exception) {
return Response.status(404).build();
}

}

Jersey behaved very similarly to CXF in terms of roundtrip time.


However, without this class, Jersey performed even faster - most of
the times were 10 times faster, i.e. around 0.003 seconds, while in
the CXF case ... you already know it :) - 0.500 seconds



Forgot to comment to this one... I think the fact that both frameworks
show approximately the same time with SimpleApplicationExceptionMapper
in place and that Jersey scales really well without it is that Jersey
basically either writes to HTTP out stream or propagates the exception
up to the ServletException filter immediately if no custom mapper is
available.
In case of CXF the immediate propagation will also happen if no custom
mapper is available. The reason the default WebApplicationException
Mapper was installed was to do with the fact that few users were not
quite happy with having to write their custom mappers in order to report
differently to the exceptions generated at the runtime level or some of
the providers' level or indeed having to write a filter catching the
escaped exceptions there...

What I might need to do additionally is to introduce a contextual
property that will tell the runtime to ignore even the default
WebApplicationException mapper, for example, when the user wishes to
get all the exceptions propagated to the filter level, to simplify the
performance data comparison, etc

Cheers, Sergey



Thank a lot for your idea and suggestion! It definitely shed light on
where the discrepancies come from.


I've opened
https://issues.apache.org/jira/browse/CXF-4121

I suspect now that it is the specific lines which get the basic warning
message from the resource bundle are main 'culprits', I guess I will
just have an internal final static String - will need to experiment a
bit...It is good even the default providers can be customized/replaced,
but I'll look into trying to make the default exception mapper
performing a bit better :-)

Thanks, Sergey


Kindest Regards,
Krum.

-----Original Message-----
From: Sergey Beryozkin [mailto:[email protected]]
Sent: Friday, February 17, 2012 6:14 PM
To: [email protected]
Subject: Re: REST performance discrepancies between CXF and Jersey

Hi Krum,
On 17/02/12 15:26, Bakalsky, Krum wrote:

Hi to all Apache CXF users!,

I would kindly like to bring to your attention a particular problem
that we are currently being stuck into. We are experimenting with a
simple web application, which exposes its functionality via REST. We
are using JAX-RS/JSON, no SOAP, no JAX-WS. We deploy our application
on top of Tomcat 7, and test both against CXF 2.5.2 and Jersey as
REST frameworks.

For calculating our performance measurements, we simply execute REST
requests, and in the client we measure the time that the roundtrip
takes. Apart from that, on the server side, we measure as well the
time that our REST resource consumes, i.e. the method call time. In
summary, we have some huge differences between CXF and Jersey - in
favor of Jersey being much faster - in the cases where we test
against non-existing resources. In all other cases, the numbers we
got were really very close for the two REST frameworks.

But when testing against non-existing REST paths, the roundtrip time
in CXF case is approximately 0.5 seconds, while in the case of Jersey
it is 0.01 seconds. We think that this is quite strange, and probably
we are not using CXF in its proper configuration. Could you please
give us some hints ? Maybe we have to tune and tweak it a little bit,
so that we get the best out of it.

The server side time, in the case of CXF, is rather small - 0.001
seconds, so we wonder where is that half second spent in ? In our
application logic, in the case when the path is not correct, we throw
javax.ws.rs.WebApplicationException and with a debugger we saw that
this exception is getting caught in CXF, and heavily processed
further down the stack.

Are you aware of such kind of issues, with CXF handling the 404
situation in a very heavyweight manner ? Maybe it does some thorough
processing to try to map the URI to some existing REST resource, I
don't know ... maybe performance here could depend on the particular
JSON framework that is being used. ...



I suspect it could be the default WebApplicationExceptionMapper
which is
to blame.
In CXF, a given Exception, once wrapped in a Response, is treated like
any other regular response, but I'm not sure it would explain the
difference.

Can you give me a favor please and register a custom
WebApplicationException mapper which would only return:

Response.status(exception.getStatus()).build() ?

Have a look please at the default mapper's code:
http://svn.apache.org/repos/asf/cxf/trunk/rt/frontend/jaxrs/src/main/java/org/apache/cxf/jaxrs/impl/WebApplicationExceptionMapper.java



As you can see, we are trying to go to some length there to figure out
what and how to report a given exception and I reckon it all adds
up to
the response time.

Please try eliminating the default mapper and let us know the results

Thanks, Sergey



Kindest Regards,
Krum.



















--
Sergey Beryozkin

Talend Community Coders
http://coders.talend.com/

Blog: http://sberyozkin.blogspot.com

Reply via email to