olegk wrote:
>
> Arash Joorabchi wrote:
>>
>>
>> olegk wrote:
>>> Arash Joorabchi wrote:
>>>> Hi all,
>>>>
>>>> I am using MultiThreadedHttpConnectionManager in httpClient 3.1 to send
>>>> 10
>>>> get methods to the Library of Congress SRU server
>>>> (http://z3950.loc.gov:7090/) concurrently each time. After running
>>>> about
>>>> 35
>>>> queries I get “server failed to respond” exceptions. In this setting I
>>>> do
>>>> not set the httpClient proxy parameters directly and instead use the
>>>> following lines in the applications’ main method
>>>>
>>>> System.getProperties().put( "proxySet", config.get_proxySet() );
>>>> System.getProperties().put( "proxyHost", config.get_proxyHost() );
>>>> System.getProperties().put( "proxyPort", config.get_proxyPort() );
>>>>
>>>> If I set the httpClient proxy parameters directly instead of “server
>>>> failed
>>>> to respond” exception I get this exception: “HTTPclinet.fetch Method
>>>> failed:
>>>> HTTP/1.1 502 Proxy Error (The specified network name is no longer
>>>> available.
>>>> )”.
>>>>
>>>> When this occurs even the Internet Explorer fails to fetch the
>>>> specified
>>>> URI
>>>> (e.g.
>>>> http://z3950.loc.gov:7090/voyager?version=1.1&operation=searchRetrieve&maximumRecords=1000&query=title=%22Nieuw+Archief+voor+Wiskunde%22
>>>> ),
>>>>
>>>> however it still works with other URIs like Google.
>>>>
>>>> To prevent this problem I have to send about 10 queries wait for 10
>>>> seconds
>>>> and then send another 10 queries. Using this setting I ran about
>>>> 20,000
>>>> queries overnight successfully.
>>>>
>>>> The most puzzling is that if I do not change the URL and send the same
>>>> query
>>>> over and over again no exception happens.
>>>>
>>>> At this point I only can think of two things happening here:
>>>>
>>>> 1. My proxy server does not respond after a while because it thinks I
>>>> am
>>>> an
>>>> internal DOS attacker.
>>>>
>>>> 2. The server thinks I am a DOS attacker and does not respond if I send
>>>> the
>>>> queries to fast.
>>>>
>>>> I would appreciate your suggestions.
>>>>
>>>> Notes:
>>>>
>>>> 1. I do release the connections after each use.
>>>> 2. I have tried sending queries with the connection close header.
>>>> 3. I also tried another setting where I run 10 queries close/release
>>>> everything shutdown the connection manager and create a new one and
>>>> send
>>>> another 10 queries.
>>>>
>>>> Thanks,
>>>> Arash Joorabchi
>>>>
>>> Arash,
>>>
>>> Whatever the reason this does sound like a server side problem to me.
>>> Consider reducing the rate of requests per time unit.
>>>
>>> Oleg
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [email protected]
>>> For additional commands, e-mail: [email protected]
>>>
>>>
>>>
>>
>> Hi Oleg,
>>
>> Thanks for your reply,
>>
>> As I mentioned in the original email If I start 10 getMethod threads,
>> wait
>> for all of them to finish and wait for another 10 seconds before starting
>> another 10 getMethod threads, then no exception is thrown, but if I
>> reduce
>> the waiting time to less than 10 seconds then I will get the following
>> exception:
>> “HTTPclinet.fetch Method failed: HTTP/1.1 502 Proxy Error (The specified
>> network name is no longer available. )” or “Proxy timeout error”.
>>
>> Do you think this is caused by my proxy server or the remote server that
>> I
>> am sending queries to?
>>
>
> In my opinion the issue is very likely to be caused by the target server.
>
> Oleg
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]
>
>
>
Hi Oleg,
You are right, I tested my application using a DMZ IP today and the same
thing happened again. Now, I am wondering if this is the cause of a fault on
the target server or a deliberate limitation set on the server to prevent
Denial Of Service/flooding attacks. However the latter seems unlikely
because I usually have 6-10 concurrently established connections and they
are kept alive to send new queries. I have contacted the Library of Congress
web master to see if they know what is happening.
Thanks,
Arash
--
View this message in context:
http://www.nabble.com/problem-with-multithreading-and-proxy-tp21912219p21960273.html
Sent from the HttpClient-User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
