Thanks for the insight Kent. The problem wasn't because of the httpOnly cookies, I just wasn't requesting the right page. I did get it to send me the httpOnly cookies when I requested the right page. Apparently, the right page was a redirection of the original page. When I used to use cURL in PHP I remember that it had an option to automatically follow redirects. How can I do this with urllib2?
For the headers, I did verify that it sent the headers as I had formed them (just not in the right order) with wireshark. So my method does work, I also got this method from here: http://www.voidspace.org.uk/python/articles/urllib2.shtml#headers On Fri, Oct 17, 2008 at 7:29 AM, Kent Johnson <[EMAIL PROTECTED]> wrote: > On Thu, Oct 16, 2008 at 11:40 PM, xbmuncher <[EMAIL PROTECTED]> wrote: > > I'm trying to mimic my firefox browser in requesting a webpage with > python. > > > So I tried trusty ol' urllib2 to request it in python: > > import urllib2 > > > > > > url = 'http://www.website.com' > > > > #headers > > h = { > > 'User-Agent' : 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv: > 1.9.0.3) > <snip> > > } > > #request page > > reqObj = urllib2.Request(url, None, h) > > urlObj = urllib2.urlopen(reqObj) > > It doesn't work to set the User-Agent header this way. See > http://personalpages.tds.net/~kent37/kk/00010.html#e10request-headers<http://personalpages.tds.net/%7Ekent37/kk/00010.html#e10request-headers> > for a recipe. > > > #read response > > print urlObj.read() > > What content do you get? Is it an error message? What does wireshark > show for this request? > > > Notice the content length is considerably smaller, and no cookies are > sent > > to me like they were in firefox. I know only a little about httpOnly > > cookies, but that it is some kind of special cookie that I suppose has > > something to do with python not being able to access it like firefox. All > I > > want to do is have python receive the same cookies that firefox did, how > can > > I do this? I read somewhere that httpOnly cookies were implemented in the > > python cookie module: > > http://glyphobet.net/blog/blurb/285 > > ....yet the other cookies aren't being sent either... > > I don't think that has anything to do with your problem. httpOnly is > set by the server and interpreted by the browser. The Python change > was to allow httpOnly to be set by servers written in Python. > > Kent >
_______________________________________________ Tutor maillist - [email protected] http://mail.python.org/mailman/listinfo/tutor
