Hi Ludovic
No problems, thanks for letting us know; and glad to see that you are
putting the server under heavy load with some tests, do let us know if you
encounter any other problems.
Regards
Steve
-----Original Message-----
From: Ludovic Deravet [mailto:ludovicdera...@gmail.com]
Sent: 09 March 2011 15:20
To: Support and info exchange list for Fedora users.
Subject: Re: [fcrepo-user] Getting a 'I/O Error: Stream closed" error on
heavy load with REST API
Hello,
Thank you for your quick replies and sorry for being late to respond to them
but I wanted to perform various intensive tests before coming back with
eventual log files.
In our Java application, we are using the HTTPClient library to generate
HTTP calls on the Fedora's REST API (setup using SpringFramework 3+) that
has the following default configuration:
41 DEBUG org.apache.commons.httpclient.params.DefaultHttpParams - Set
parameter http.useragent = Jakarta Commons-HttpClient/3.1
43 DEBUG org.apache.commons.httpclient.params.DefaultHttpParams - Set
parameter http.protocol.version = HTTP/1.1
44 DEBUG org.apache.commons.httpclient.params.DefaultHttpParams - Set
parameter http.connection-manager.class = class
org.apache.commons.httpclient.SimpleHttpConnectionManager
44 DEBUG org.apache.commons.httpclient.params.DefaultHttpParams - Set
parameter http.protocol.cookie-policy = default
44 DEBUG org.apache.commons.httpclient.params.DefaultHttpParams - Set
parameter http.protocol.element-charset = US-ASCII
44 DEBUG org.apache.commons.httpclient.params.DefaultHttpParams - Set
parameter http.protocol.content-charset = ISO-8859-1
(the class highlighted in red was the cause of the problem in multi-threads
context).
In this default configuration, the HTTPConnection instance was shared among
all threads and was generating the following kind of errors (connection is
not open, attempted read on closed stream, no Content-Type found) when the
application is under heavy load.
Indeed, in our situation, we are ingesting hundred of METS envelopes (and so
creating thousands of digital objects with access to datastreams through the
Fedora's REST API) in few minutes. It was only in that case that the
connections to the Fedora server were failing randomly.
So in summary, with Fedora 3.4.2, everything works fine when using
MultiThreadedHttpConnectionManager (of HttpClient) when calling the REST
API.
Sorry for the misleading message that might have raised doubts about the
possible errors under heavy load.
Thanks again for your support.
Kind regards,
Ludovic
On Wed, Mar 9, 2011 at 10:48 AM, Asger Askov Blekinge
<a...@statsbiblioteket.dk> wrote:
Hi
I was the guy that solved the previous bugs. Give me the log files, and I
will solve this one for you.
Regards
On 2011-03-08 12:42, Steve Bayliss wrote:
Hi Ludovic
It would be great if you're able to reproduce this and supply the Fedora
server log file so we can try and track down exactly where this is happening
in the code.
Regards
Steve
-----Original Message-----
From: Ludovic Deravet [mailto:ludovicdera...@gmail.com]
Sent: 08 March 2011 08:08
To: Support and info exchange list for Fedora users.
Subject: Re: [fcrepo-user] Getting a 'I/O Error: Stream closed" error on
heavy load with REST API
Hello Scott,
We have migrated to 3.4.2 a week ago so we are getting this problem on this
release.
Kind regards,
Ludovic
On Tue, Mar 8, 2011 at 1:01 AM, Scott Prater <pra...@wisc.edu> wrote:
Ludovic,
What version of Fedora are you on? It looks like you may have run into
this bug:
https://jira.duraspace.org/browse/FCREPO-774
... which was fixed in version 3.4.2.
-- Scott
Ludovic Deravet wrote:
> Hello,
>
> We are having a strange issue (*Stream closed*) when working for the REST
> API of Fedora:
>
> 2011-03-07 23:23:15,864 ERROR [DISSEMINATION] (http-127.0.0.1-8080-4)
> Exception while processing dissemination service request:
>
uri=/resource/709d8fa2-592a-4d3e-a86d-c1279741a41c;headers=[host=localhost:8
080,
> user-agent=Mozilla/5.0 (Windows; U; Windows NT 6.1; fr; rv:1.9.2.6)
> Gecko/20100625 Firefox/3.6.6,
> accept-language=fr,fr-fr;q=0.8,en-us;q=0.5,en;q=0.3,
> accept-encoding=gzip,deflate,
accept-charset=ISO-8859-1,utf-8;q=0.7,*;q=0.7,
> keep-alive=115, connection=keep-alive, cookie=GUEST_LANGUAGE_ID=en_US;
> COOKIE_SUPPORT=true; LOGIN=746573744074656e666f7263652e636f6d;
> SCREEN_NAME=6f6672492f4353617541493d, accept=application/xml;notice=tree,
> cache-control=max-age=0]. [ps 'cellar' - psid
> '709d8fa2-592a-4d3e-a86d-c1279741a41c']: I/O error: Stream closed; nested
> exception is java.io.IOException: Stream closed
>
> org.springframework.web.client.ResourceAccessException: I/O error: Stream
> closed; nested exception is java.io.IOException: Stream closed
>
> at
>
org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:453)
>
> at
> org.springframework.web.client.RestTemplate.execute(RestTemplate.java:401)
>
> at
>
org.springframework.web.client.RestTemplate.getForObject(RestTemplate.java:1
99)
>
>
> The Springframework class RestTemplate is call by our code and looks like
> this:
>
> restTemplate.getForObject(fedoraUrl, String.class, urlVariables);
>
> It runs fine under normal load but start to appears randomly under heavy
> load. Someone has already got such a problem and can give me some hints
> about how to solve it? Thanks.
>
> Kind regards
>
>
> ------------------------------------------------------------------------
>
>
----------------------------------------------------------------------------
--
> What You Don't Know About Data Connectivity CAN Hurt You
> This paper provides an overview of data connectivity, details
> its effect on application quality, and explores various alternative
> solutions. http://p.sf.net/sfu/progress-d2d
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Fedora-commons-users mailing list
> Fedora-commons-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/fedora-commons-users
--
Scott Prater
Library, Instructional, and Research Applications (LIRA)
Division of Information Technology (DoIT)
University of Wisconsin - Madison
pra...@wisc.edu
----------------------------------------------------------------------------
--
What You Don't Know About Data Connectivity CAN Hurt You
This paper provides an overview of data connectivity, details
its effect on application quality, and explores various alternative
solutions. http://p.sf.net/sfu/progress-d2d
_______________________________________________
Fedora-commons-users mailing list
Fedora-commons-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/fedora-commons-users
--
Ludovic
----------------------------------------------------------------------------
--
Colocation vs. Managed Hosting
A question and answer guide to determining the best fit
for your organization - today and in the future.
http://p.sf.net/sfu/internap-sfd2d
_______________________________________________
Fedora-commons-users mailing list
Fedora-commons-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/fedora-commons-users
--
Ludovic
------------------------------------------------------------------------------
Colocation vs. Managed Hosting
A question and answer guide to determining the best fit
for your organization - today and in the future.
http://p.sf.net/sfu/internap-sfd2d
_______________________________________________
Fedora-commons-users mailing list
Fedora-commons-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/fedora-commons-users
