Hi
I have the need to stream large XML responses back to the client using
Jax-RS. We have a large number of objects (Potentially upto a million) which
need to be marshalled and the response returned, is the support for
streaming XML responses while objects are being marshalled in CXF at the
moment? We are currently seeing some large degradation in performance at
times when these large number of entities are being marshalled.
I basically return the objects which need to be marshalled from our Service
methods, what would need to change for me to be able to make use of the
streaming support?
I can think of few options. I do believe the CXF runtime has all what is needed to do the effective streaming back to the client but
I will need to ask Dan for some clarifications, some updates might need to be applied to CXF JAXRS.
As far as JAXRS itself is concerned, you might want to choose to return an instance of StreamingOutput from a method. Or JAXP Source
and actually return an instance of CXF StaxSource.
If it is JAXB that you use then you may want to try explicitly registering JAXBElementProvider and setting an "enableStreaming"
boolean property on it in which case JAXBProvider will create an XMLStreamWriter and pass it to JAXB Marshaller. This option looks
similar to explicitly returning an instance of StaxSource.
Another option is to return a multipart formatted response, please see :
http://cwiki.apache.org/CXF20DOC/jax-rs.html#JAX-RS-Writingattachments
Another option which might be worth evaluating is to return a list of links back to a client (embedded in some minimal custom XML
instance) so that a client can fetch data from different links in parallel which might improve the overall experience...Similar
option is to update the interface for it to support the pagination...
Let me know please what do you think is the best option for your project and then we can focus on ensuring that option is supported
well by CXF JAXRS
thanks, Sergey
-----Original Message-----
From: Sergey Beryozkin [mailto:[email protected]]
Sent: Friday, 9 October 2009 11:12 PM
To: [email protected]
Cc: rsmith
Subject: Re: CXF and large XML request/responses : streaming support?
Hi
It is interesting, especially the Stax support.I'm not familiar with the
recent build of CXF, on this matter would it be also available for the
JAX-RS support.
I missed it...I think in the case of JAXRS declaring a method accepting
(JAXP) Source will work once
I update a SourceProvider to check if XMLStreamReader is available on the
message (or create a new one if it is a multipart request)
and then wrap it in StaxSource and just pass it on - will be done for 2.3;
if you need it working now then I can help you with
creating a custom SourceProvider...The existing MultipartProvider will just
delegate to it.
thanks, Sergey
Anyway great framework :)
On Thu, Oct 8, 2009 at 19:29, Daniel Kulp <[email protected]> wrote:
Right now, with a JAX-WS provider, there is SOME support for this, but
its
far
from ideal. This is an area I'll be working in next week (resolving
customer
issues) and I'll see if I can add some enhancements easily enough.
Basically, right now, if you do Provider<Source>, you would get
DOMSource
in
(thus, the incoming message would not be streamed, but you could return a
StreamSource or SAXSource orsimilar to use that we would use to copy
stuff
out. If you did Provider<StreamSource> or Provider<SAXSource>, we pull
the
full message into a Cached stream (which, for large messages, would
output
to
temp files on disks) and return that to you. Thus, the whole thing
isn't
in
memory, but it does result in the temp files and such.
Part of what I hope to do next week is enable:
Provider<XMLStreamReader>
and/or
Provider<StaxSource>
which would allow full streaming in most cases.
Dan
On Wed October 7 2009 12:37:50 am rsmith wrote:
> I'm trying to find out if CXF supports full streaming of input and
> output
> messages for the SOAP transport.
>
> I have a service that will be receiving large input XML payload, and
> will
> be generating a response with a large XML payload. I can process the
> input XML incrementally, generating the response as the input is
> processed.
>
> Is there a way to implement a service in CXF streaming at all levels
> (XML
> parsing, data binding, generating response), avoiding holding the full
> document in memory at any time?
>
> I found several threads on the mailing list, some of which make it
> sound
> like it's supported. This message gave me the impression it may not
> currently be supported though:
>
http://www.nabble.com/Re%3A-Configuring-streaming-web-services%3A-error-on-
> the-call-to-invoke-p24187339.html
>
> Some of the other threads:
>
http://www.nabble.com/Looking-for-a-solution-for-Large-XML-Messages---strea
> ming-and-JAXWS-td20451942.html#a20451942
>
http://www.nabble.com/Recommended-way-to-have-a-web-method-stream-results-
> back-to-client--td22856243.html#a22864087
> http://www.nabble.com/SAXSource-td24411461.html#a24411461
>
> Thanks in advance
>
--
Daniel Kulp
[email protected]
http://www.dankulp.com/blog
--
Bryce
