Hey Sebb, Thnx alot. I used a HTTP sniffer and found out that the requests from my client were not correct. Problem is solved now :) Micky
________________________________ Van: sebb [mailto:[EMAIL PROTECTED] Verzonden: di 20-5-2008 15:22 Aan: HttpClient User Discussion Onderwerp: Re: HttpClient returns different response compared to browser On 20/05/2008, Kwik, Micky <[EMAIL PROTECTED]> wrote: > Hi, > > I wrote a simple client to fetch documents from some websites. But I found > that the HttpClient often gets a different response compared to the browser > even if the HTTP status code is 200. In what way is the response different? > For example this URL: > http://www.elsevierfiscaal.nl/els/enc/productserviceoverzicht/id1101-31813/search/true/channelId/1101/update-14-aangifte-assistent-2008.html > > <https://webmail.sogeti.nl/exchweb/bin/redir.asp?URL=http://www.elsevierfiscaal.nl/els/enc/productserviceoverzicht/id1101-31813/search/true/channelId/1101/update-14-aangifte-assistent-2008.html> > or > http://www.belastingdienst.nl/zakelijk/nieuwsbrief/nieuwsberichten/2008-04-02-08_franke.html > > <https://webmail.sogeti.nl/exchweb/bin/redir.asp?URL=http://www.belastingdienst.nl/zakelijk/nieuwsbrief/nieuwsberichten/2008-04-02-08_franke.html> > > Here is my code snippet: > HttpClient client = setUpClient(aUrl); > GetMethod method = new GetMethod(); > method.getParams().setParameter("http.useragent", "Mozilla/5.0 (Windows; U; > Windows NT 5.1; en-US; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14"); > > method.getParams().setBooleanParameter("http.protocol.single-cookie-header", > true); > method.setFollowRedirects(false); > method.getParams().setCookiePolicy(CookiePolicy.BROWSER_COMPATIBILITY); > > int returnCode = client.executeMethod(method); > if (returnCode == HttpStatus.SC_OK) { > fetchBody(aUrl, method); > } > I already varied Cookie policies and followRedirects but with no success. Is > there a way of solving this or is it a case of "http client is not a browser" > ? So long as HttpClient is set up to send the correct HTTP requests, the server cannot tell if it is talking to a browser or not. [Well, I suppose it could do some clever tricks with timing or Javascript. But that is unlikely to be the case here.] So you just need to find out what the difference is between what the browser sends and what HttpClient is sending. There may be some extra hidden fields or other parameters that have been overlooked. A protocol sniffer such as Wireshark - or a recording proxy - would be helpful here. > Kind regards, > Micky > > Disclaimer: > This message contains information that may be privileged or confidential and > is the property of Sogeti Nederland B.V. or its Group members. It is intended > only for the person to whom it is addressed. If you are not the intended > recipient, you are not authorized to read, print, retain, copy, disseminate, > distribute, or use this message or any part thereof. If you receive this > message in error, please notify the sender immediately and delete all copies > of this message. > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] Disclaimer: This message contains information that may be privileged or confidential and is the property of Sogeti Nederland B.V. or its Group members. It is intended only for the person to whom it is addressed. If you are not the intended recipient, you are not authorized to read, print, retain, copy, disseminate, distribute, or use this message or any part thereof. If you receive this message in error, please notify the sender immediately and delete all copies of this message.
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
