Thanks Greg - this should not be happening this way. I've raised https://jira.duraspace.org/browse/FCREPO-1020 for this.
Steve > -----Original Message----- > From: Greg Jansen [mailto:cou...@email.unc.edu] > Sent: 24 October 2011 14:50 > To: 'fedora-dev net sourceforge.' > Subject: [fcrepo-dev] PEP.java and problems with large disseminations > > > Hey folks, > I've been seeing long delays and memory filling up when Fedora is > disseminating a large data streams. The same behavior may also be > present on smaller streams, but is not as noticeable. We have > some 2GB > Wave files in our repository, which will trigger OOM exceptions if > downloaded this way. We are using FeSL here and from looking over the > relevant Fedora code, it looks like the PEP.java servlet > filter class is > wrapping and potentially buffering the whole request in > memory via the > DataResponseWrapper. > > <snip> > out.write(res.getData()); // returns a byte array containing the > whole request > out.flush(); > </snip> > > Is this the expected pathway for a data stream dissemination > request? It > seems like even a couple simultaneous small files could fill > the heap. > This makes me think I've configured it improperly somehow. > > thanks for your help, > Greg Jansen > > p.s. > Here is the stack trace that I generally get if I trigger > this when the > JVM isn't doing anything else: > > SEVERE: Servlet.service() for servlet RestServlet threw exception > java.lang.OutOfMemoryError: Java heap space > at java.util.Arrays.copyOf(Arrays.java:2786) > at > java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94) > at java.io.DataOutputStream.write(DataOutputStream.java:90) > at > org.fcrepo.server.security.xacml.pep.rest.filters.DataServletO > utputStream.write(DataServletOutputStream.java:71) > at > com.sun.jersey.spi.container.servlet.WebComponent$Writer.write > (WebComponent.java:230) > at > com.sun.jersey.spi.container.ContainerResponse$CommittingOutpu > tStream.write(ContainerResponse.java:114) > at > com.sun.jersey.core.provider.AbstractMessageReaderWriterProvid > er.writeTo(AbstractMessageReaderWriterProvider.java:73) > at > com.sun.jersey.core.impl.provider.entity.InputStreamProvider.w > riteTo(InputStreamProvider.java:95) > at > com.sun.jersey.core.impl.provider.entity.InputStreamProvider.w > riteTo(InputStreamProvider.java:58) > at > com.sun.jersey.spi.container.ContainerResponse.write(Container > Response.java:254) > at > com.sun.jersey.server.impl.application.WebApplicationImpl._han > dleRequest(WebApplicationImpl.java:724) > at > com.sun.jersey.server.impl.application.WebApplicationImpl.hand > leRequest(WebApplicationImpl.java:647) > at > com.sun.jersey.server.impl.application.WebApplicationImpl.hand > leRequest(WebApplicationImpl.java:638) > at > com.sun.jersey.spi.container.servlet.WebComponent.service(WebC > omponent.java:309) > at > com.sun.jersey.spi.container.servlet.ServletContainer.service( > ServletContainer.java:425) > at > com.sun.jersey.spi.container.servlet.ServletContainer.service( > ServletContainer.java:590) > at > javax.servlet.http.HttpServlet.service(HttpServlet.java:717) > > -- > ___ > Gregory N. Jansen > Developer - Carolina Digital Repository > UNC Chapel Hill Libraries > > > -------------------------------------------------------------- > ---------------- > The demand for IT networking professionals continues to grow, > and the demand for specialized networking skills is growing > even more rapidly. Take a complimentary Learning@Cisco > Self-Assessment and learn > about Cisco certifications, training, and career opportunities. > http://p.sf.net/sfu/cisco-dev2dev > _______________________________________________ > Fedora-commons-developers mailing list > Fedora-commons-developers@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/fedora-commons-developers > ------------------------------------------------------------------------------ The demand for IT networking professionals continues to grow, and the demand for specialized networking skills is growing even more rapidly. Take a complimentary Learning@Cisco Self-Assessment and learn about Cisco certifications, training, and career opportunities. http://p.sf.net/sfu/cisco-dev2dev _______________________________________________ Fedora-commons-developers mailing list Fedora-commons-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/fedora-commons-developers
