Re: [BangPypers] Auto Login to Website and Capturing more details
Hi
Try requests, this is possible with it, use authentications in requests.
On Tue, Dec 27, 2011 at 9:28 PM, Nikunj Badjatya
nikunjbadja...@gmail.comwrote:
Noufal,
I tried with :
{{{
import urllib, urllib2, cookielib
username = 'xxx'
password = 'xxx'
savepage = open('output.html', 'w')
#cj = cookielib.CookieJar()
#opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor())
login_data = urllib.urlencode({'username' : username, 'j_password' :
password})
opener.open('https://ebpp.airtelworld.com/pkmslogin.form', login_data)
resp = opener.open('
https://ebpp.airtelworld.com/cares/wps/myportal/!ut/p/.cmd/cs/.ce/7_0_A/.s/7_0_J5/_s.7_0_A/7_0_J5
')
savepage.write(resp.read())
savepage.close()
}}}
The link in the 'resp' is true to my knowledge. I checked it by logging in
at diff times. Its coming the same every time and it is the link behind
'Account Information' which comes on the front page after logging in.
When I open output.html, Its giving me the same login form i.e same looks
as those of 'https://ebpp.airtelworld.com/pkmslogin.form'
I tried with CookieJar() also. Same results.
Any ideas?
Thanks,
Nikunj
--
*7*Switch off as you go |*q*Recycle always | P Save Paper - Save Trees | Go
Green
On Fri, Dec 23, 2011 at 1:29 AM, Saager Mhatre saager.mha...@gmail.com
wrote:
On Thu, Dec 22, 2011 at 10:27 PM, Noufal Ibrahim nou...@gmail.com
wrote:
[snip]
The Python stdlib can handle this using urllib2 and cookielib. I've
managed
something like this here
https://gist.github.com/3811b566df4005a012c7which you can use as a
starting point.
After this, you have a session. Now you should be able to GET the URL
you
want directly and parse it for the information you need.
Wouldn't something like mechanize
http://pypi.python.org/pypi/mechanizesimplify all this?
- d
___
BangPypers mailing list
BangPypers@python.org
http://mail.python.org/mailman/listinfo/bangpypers
___
BangPypers mailing list
BangPypers@python.org
http://mail.python.org/mailman/listinfo/bangpypers
--
*
Thanks Regards
Talk is cheap, show me the code -- Linus Torvalds
kracekumar
www.kracekumar.com
*
___
BangPypers mailing list
BangPypers@python.org
http://mail.python.org/mailman/listinfo/bangpypers
Re: [BangPypers] Auto Login to Website and Capturing more details
Noufal,
I tried with :
{{{
import urllib, urllib2, cookielib
username = 'xxx'
password = 'xxx'
savepage = open('output.html', 'w')
#cj = cookielib.CookieJar()
#opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor())
login_data = urllib.urlencode({'username' : username, 'j_password' :
password})
opener.open('https://ebpp.airtelworld.com/pkmslogin.form', login_data)
resp = opener.open('
https://ebpp.airtelworld.com/cares/wps/myportal/!ut/p/.cmd/cs/.ce/7_0_A/.s/7_0_J5/_s.7_0_A/7_0_J5
')
savepage.write(resp.read())
savepage.close()
}}}
The link in the 'resp' is true to my knowledge. I checked it by logging in
at diff times. Its coming the same every time and it is the link behind
'Account Information' which comes on the front page after logging in.
When I open output.html, Its giving me the same login form i.e same looks
as those of 'https://ebpp.airtelworld.com/pkmslogin.form'
I tried with CookieJar() also. Same results.
Any ideas?
Thanks,
Nikunj
--
*7*Switch off as you go |*q*Recycle always | P Save Paper - Save Trees | Go
Green
On Fri, Dec 23, 2011 at 1:29 AM, Saager Mhatre saager.mha...@gmail.comwrote:
On Thu, Dec 22, 2011 at 10:27 PM, Noufal Ibrahim nou...@gmail.com wrote:
[snip]
The Python stdlib can handle this using urllib2 and cookielib. I've managed
something like this here
https://gist.github.com/3811b566df4005a012c7which you can use as a
starting point.
After this, you have a session. Now you should be able to GET the URL you
want directly and parse it for the information you need.
Wouldn't something like mechanize
http://pypi.python.org/pypi/mechanizesimplify all this?
- d
___
BangPypers mailing list
BangPypers@python.org
http://mail.python.org/mailman/listinfo/bangpypers
___
BangPypers mailing list
BangPypers@python.org
http://mail.python.org/mailman/listinfo/bangpypers
Re: [BangPypers] Auto Login to Website and Capturing more details
Nikunj Badjatya nikunjbadja...@gmail.com writes: [...] The link in the 'resp' is true to my knowledge. I checked it by logging in at diff times. Its coming the same every time and it is the link behind 'Account Information' which comes on the front page after logging in. When I open output.html, Its giving me the same login form i.e same looks as those of 'https://ebpp.airtelworld.com/pkmslogin.form' I tried with CookieJar() also. Same results. Can't say for sure. Are you getting back the session cookie? It might also make sense to try a more modern HTTP client library rather then stdlib. Krace's suggestion of requests is good as is Sageer's one of Mechanize. [...] -- ~noufal http://nibrahim.net.in Some bachelors want a meaningful overnight relationship. ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers
Re: [BangPypers] Auto Login to Website and Capturing more details
Alternatively, try Sahi. (http://sahi.co.in). There's an open source version available for download and use. -- Ram On 12/27/11, Noufal Ibrahim nou...@gmail.com wrote: Nikunj Badjatya nikunjbadja...@gmail.com writes: [...] The link in the 'resp' is true to my knowledge. I checked it by logging in at diff times. Its coming the same every time and it is the link behind 'Account Information' which comes on the front page after logging in. When I open output.html, Its giving me the same login form i.e same looks as those of 'https://ebpp.airtelworld.com/pkmslogin.form' I tried with CookieJar() also. Same results. Can't say for sure. Are you getting back the session cookie? It might also make sense to try a more modern HTTP client library rather then stdlib. Krace's suggestion of requests is good as is Sageer's one of Mechanize. [...] -- ~noufal http://nibrahim.net.in Some bachelors want a meaningful overnight relationship. ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers -- Sent from my mobile device Belenix: www.belenix.org Twitter: @sriramnrn ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers
Re: [BangPypers] Auto Login to Website and Capturing more details
Nikunj Badjatya nikunjbadja...@gmail.com writes: Hi All, I am trying to use Python power in my daily life. Do you have Airtel Broadband Connection ? YES / NO ? In either case..read on. ! I am trying to write a script which will automatically login to Airtel page https://ebpp.airtelworld.com/cares/wps/myportal Then go to account information - DSL Account - View Unbilled Information And prints the details of unbilled information. Can anyone guide me on this.? How to start etc. The site is definitely javascript based ( Please correct if I am wrong! ) You're probably right. People just love to use JS to make their sites Web 2.0 these days but it's something we screen scrapers have to deal with. Here are my comments from a quick examination using Firebug (which is something you should have handy to investigate stuff like this). The login attempt POSTs your data to https://ebpp.airtelworld.com/pkmslogin.form which returns a secure cookie that should probably be sent in subsequent requests to handle your session. This is probably the first thing you need to do. The Python stdlib can handle this using urllib2 and cookielib. I've managed something like this here https://gist.github.com/3811b566df4005a012c7 which you can use as a starting point. After this, you have a session. Now you should be able to GET the URL you want directly and parse it for the information you need. [...] -- ~noufal http://nibrahim.net.in Evil isn't all bad. ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers
Re: [BangPypers] Auto Login to Website and Capturing more details
On Thu, Dec 22, 2011 at 10:27 PM, Noufal Ibrahim nou...@gmail.com wrote: [snip] The Python stdlib can handle this using urllib2 and cookielib. I've managed something like this here https://gist.github.com/3811b566df4005a012c7which you can use as a starting point. After this, you have a session. Now you should be able to GET the URL you want directly and parse it for the information you need. Wouldn't something like mechanize http://pypi.python.org/pypi/mechanizesimplify all this? - d ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers
