I am little new to Java. I am working on web scraping program for school. I
found out before you can web scrape, you have to create a cookie with
userid/password information. I am using HTTPClient library to do that.
Following is my code. I am having problem with httpclient. execute(httpget)
line, it just goes to error.jsp page, without even going to catch section
Please help.

my call: setAuthenticationCookie("","","http://www.msn.com ");

public void setAuthenticationCookie( String sUsername, String sPassword,
String sURL)
    {

       try{
           DefaultHttpClient client = new DefaultHttpClient();
           // set per default

           //client.getParams().setParameter(
           //    HttpClientParams.COOKIE_POLICY ,
CookiePolicy.BROWSER_COMPATIBILITY);

           HttpGet httpget = new HttpGet(sURL);
           HttpResponse response = client.execute(httpget); //PROBLEM HERE
and above too. sounds like client is not getting a valid handler (however
client does not have 0x000 pointer , it does return 213 some number, i guess
that means it points to valid address, coming from C++ background, java
confuses me)

HttpEntity entity = response.getEntity();
           if( entity != null){
                entity.consumeContent();
           }

           Cookie[] cookies = client.getState ().getCookies();


           HttpPost httppost = new HttpPost(sURL);
           NameValuePair[] valuePair = new NameValuePair[] {
                        new BasicNameValuePair("IDToken1", "username"),
                        new BasicNameValuePair("IDToken2", "password"),
                        };
           //Only for https
           //httppost.setEntity(new UrlEncodedFormEntity(valuePair,
HTTP.UTF_8));
           response = client.execute(httppost);
           entity = response.getEntity();
           if( entity != null){
                entity.consumeContent();
           }

       }
       catch (Exception e) {
            e.printStackTrace(System.err);
       }
}

Reply via email to