Hello,
Thank you for your quick replies and sorry for being late to respond to them
but I wanted to perform various intensive tests before coming back with
eventual log files.
In our Java application, we are using the HTTPClient library to generate
HTTP calls on the Fedora's REST API (setup using SpringFramework 3+) that
has the following default configuration:
41 DEBUG org.apache.commons.httpclient.params.DefaultHttpParams - Set
parameter http.useragent = Jakarta Commons-HttpClient/3.1
43 DEBUG org.apache.commons.httpclient.params.DefaultHttpParams - Set
parameter http.protocol.version = HTTP/1.1
44 DEBUG org.apache.commons.httpclient.params.DefaultHttpParams - Set
parameter http.connection-manager.class = class
org.apache.commons.httpclient.SimpleHttpConnectionManager
44 DEBUG org.apache.commons.httpclient.params.DefaultHttpParams - Set
parameter http.protocol.cookie-policy = default
44 DEBUG org.apache.commons.httpclient.params.DefaultHttpParams - Set
parameter http.protocol.element-charset = US-ASCII
44 DEBUG org.apache.commons.httpclient.params.DefaultHttpParams - Set
parameter http.protocol.content-charset = ISO-8859-1
(the class highlighted in red was the cause of the problem in multi-threads
context).
In this default configuration, the HTTPConnection instance was shared among
all threads and was generating the following kind of errors (connection is
not open, attempted read on closed stream, no Content-Type found) when the
application is under heavy load.
Indeed, in our situation, we are ingesting hundred of METS envelopes (and so
creating thousands of digital objects with access to datastreams through the
Fedora's REST API) in few minutes. It was only in that case that the
connections to the Fedora server were failing randomly.
So in summary, with Fedora 3.4.2, everything works fine when using
MultiThreadedHttpConnectionManager
(of HttpClient) when calling the REST API.
Sorry for the misleading message that might have raised doubts about the
possible errors under heavy load.
Thanks again for your support.
Kind regards,
*Ludovic*
On Wed, Mar 9, 2011 at 10:48 AM, Asger Askov Blekinge <
a...@statsbiblioteket.dk> wrote:
> Hi
>
> I was the guy that solved the previous bugs. Give me the log files, and I
> will solve this one for you.
>
> Regards
>
>
> On 2011-03-08 12:42, Steve Bayliss wrote:
>
> Hi Ludovic
>
> It would be great if you're able to reproduce this and supply the Fedora
> server log file so we can try and track down exactly where this is happening
> in the code.
>
> Regards
> Steve
>
> -----Original Message-----
> *From:* Ludovic Deravet
> [mailto:ludovicdera...@gmail.com<ludovicdera...@gmail.com>]
>
> *Sent:* 08 March 2011 08:08
> *To:* Support and info exchange list for Fedora users.
> *Subject:* Re: [fcrepo-user] Getting a 'I/O Error: Stream closed" error on
> heavy load with REST API
>
> Hello Scott,
>
> We have migrated to 3.4.2 a week ago so we are getting this problem on
> this release.
>
> Kind regards,
>
> *Ludovic*
>
> On Tue, Mar 8, 2011 at 1:01 AM, Scott Prater <pra...@wisc.edu> wrote:
>
>> Ludovic,
>>
>> What version of Fedora are you on? It looks like you may have run into
>> this bug:
>>
>> https://jira.duraspace.org/browse/FCREPO-774
>>
>> ... which was fixed in version 3.4.2.
>>
>> -- Scott
>>
>>
>> Ludovic Deravet wrote:
>> > Hello,
>> >
>> > We are having a strange issue (*Stream closed*) when working for the
>> REST
>> > API of Fedora:
>> >
>> > 2011-03-07 23:23:15,864 ERROR [DISSEMINATION] (http-127.0.0.1-8080-4)
>> > Exception while processing dissemination service request:
>> >
>> uri=/resource/709d8fa2-592a-4d3e-a86d-c1279741a41c;headers=[host=localhost:8080,
>> > user-agent=Mozilla/5.0 (Windows; U; Windows NT 6.1; fr; rv:1.9.2.6)
>> > Gecko/20100625 Firefox/3.6.6,
>> > accept-language=fr,fr-fr;q=0.8,en-us;q=0.5,en;q=0.3,
>> > accept-encoding=gzip,deflate,
>> accept-charset=ISO-8859-1,utf-8;q=0.7,*;q=0.7,
>> > keep-alive=115, connection=keep-alive, cookie=GUEST_LANGUAGE_ID=en_US;
>> > COOKIE_SUPPORT=true; LOGIN=746573744074656e666f7263652e636f6d;
>> > SCREEN_NAME=6f6672492f4353617541493d,
>> accept=application/xml;notice=tree,
>> > cache-control=max-age=0]. [ps 'cellar' - psid
>> > '709d8fa2-592a-4d3e-a86d-c1279741a41c']: I/O error: Stream closed;
>> nested
>> > exception is java.io.IOException: Stream closed
>> >
>> > org.springframework.web.client.ResourceAccessException: I/O error:
>> Stream
>> > closed; nested exception is java.io.IOException: Stream closed
>> >
>> > at
>> >
>> org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:453)
>> >
>> > at
>> >
>> org.springframework.web.client.RestTemplate.execute(RestTemplate.java:401)
>> >
>> > at
>> >
>> org.springframework.web.client.RestTemplate.getForObject(RestTemplate.java:199)
>> >
>> >
>> > The Springframework class RestTemplate is call by our code and looks
>> like
>> > this:
>> >
>> > restTemplate.getForObject(fedoraUrl, String.class, urlVariables);
>> >
>> > It runs fine under normal load but start to appears randomly under heavy
>> > load. Someone has already got such a problem and can give me some hints
>> > about how to solve it? Thanks.
>> >
>> > Kind regards
>> >
>> >
>> >
>> ------------------------------------------------------------------------
>> >
>> >
>> ------------------------------------------------------------------------------
>> > What You Don't Know About Data Connectivity CAN Hurt You
>> > This paper provides an overview of data connectivity, details
>> > its effect on application quality, and explores various alternative
>> > solutions. http://p.sf.net/sfu/progress-d2d
>> >
>> >
>> > ------------------------------------------------------------------------
>> >
>> > _______________________________________________
>> > Fedora-commons-users mailing list
>> > Fedora-commons-users@lists.sourceforge.net
>> > https://lists.sourceforge.net/lists/listinfo/fedora-commons-users
>>
>>
>> --
>> Scott Prater
>> Library, Instructional, and Research Applications (LIRA)
>> Division of Information Technology (DoIT)
>> University of Wisconsin - Madison
>> pra...@wisc.edu
>>
>>
>> ------------------------------------------------------------------------------
>> What You Don't Know About Data Connectivity CAN Hurt You
>> This paper provides an overview of data connectivity, details
>> its effect on application quality, and explores various alternative
>> solutions. http://p.sf.net/sfu/progress-d2d
>> _______________________________________________
>> Fedora-commons-users mailing list
>> Fedora-commons-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/fedora-commons-users
>>
>
>
>
> --
> *Ludovic*
>
>
>
>
> ------------------------------------------------------------------------------
> Colocation vs. Managed Hosting
> A question and answer guide to determining the best fit
> for your organization - today and in the future.
> http://p.sf.net/sfu/internap-sfd2d
> _______________________________________________
> Fedora-commons-users mailing list
> Fedora-commons-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/fedora-commons-users
>
>
--
*Ludovic*
------------------------------------------------------------------------------
Colocation vs. Managed Hosting
A question and answer guide to determining the best fit
for your organization - today and in the future.
http://p.sf.net/sfu/internap-sfd2d
_______________________________________________
Fedora-commons-users mailing list
Fedora-commons-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/fedora-commons-users
