Re: Python HTTP POST
On Thu, Jul 18 2013,Joel Goldstick wrote: [snipped 28 lines] Many people find urllib and urllib2 to be confusing. There is a module called requests which makes this stuff a lot easier. ymmv http://docs.python-requests.org/en/latest/ Yes, please use this instead of the url* ones, easier to work with and tinker. I just finished writing a small scraper using it. The documentation too is very good. sivaram -- -- http://mail.python.org/mailman/listinfo/python-list
Re: Python HTTP POST
In 00ec2f9b-fcae-428c-8932-163e653dd...@googlegroups.com Matt Graves
tunacu...@gmail.com writes:
How would I submit a python HTTP POST request to... for example, go to
google.com, enter Pie into the search box and submit (Search)
Something like this:
import urllib
import urllib2
# the google form search input box is named 'q'
data = { 'q': 'Pie' }
response = urllib2.urlopen('http://google.com', urllib.urlencode(data))
print response.read()
--
John Gordon A is for Amy, who fell down the stairs
gor...@panix.com B is for Basil, assaulted by bears
-- Edward Gorey, The Gashlycrumb Tinies
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python HTTP POST
On Wed, Jul 17, 2013 at 4:15 PM, John Gordon gor...@panix.com wrote:
In 00ec2f9b-fcae-428c-8932-163e653dd...@googlegroups.com Matt Graves
tunacu...@gmail.com writes:
How would I submit a python HTTP POST request to... for example, go to
google.com, enter Pie into the search box and submit (Search)
Something like this:
import urllib
import urllib2
# the google form search input box is named 'q'
data = { 'q': 'Pie' }
response = urllib2.urlopen('http://google.com', urllib.urlencode(data))
print response.read()
--
John Gordon A is for Amy, who fell down the stairs
gor...@panix.com B is for Basil, assaulted by bears
-- Edward Gorey, The Gashlycrumb Tinies
--
http://mail.python.org/mailman/listinfo/python-list
Many people find urllib and urllib2 to be confusing. There is a module
called requests which makes this stuff a lot easier. ymmv
http://docs.python-requests.org/en/latest/
--
Joel Goldstick
http://joelgoldstick.com
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python HTTP POST
On 18/07/2013 4:49 AM, Matt Graves wrote: How would I submit a python HTTP POST request to... for example, go to google.com, enter Pie into the search box and submit (Search) Other replies have suggested how you could do it by building the request yourself. Another approach is to interact directly with the form using mechanize: http://wwwsearch.sourceforge.net/mechanize/ Unfortunately this doesn't appear to actually work with Google, as its robots.txt won't allow it. But if you're dealing with an intranet-based form in your organisation you should be fine. -- http://mail.python.org/mailman/listinfo/python-list
Re: PYTHON HTTP POST WORKING EXAMPLE NEEDED
On Fri, Jan 23, 2009 at 8:02 PM, r rt8...@gmail.com wrote: Whatever scumbag would rate a new python's users with one start for asking us for help is a complete low-life. Keep acting like that and Python will fade away into the darkness forever. Could you please explain where these stars you keep talking about come from? It's not from Usenet (AFAIKT), the mailing list, or Google Groups. Stop worrying about something that 99% of the people here can't see. dhaval, Don't let the few bullies here scare you away, try out the this link, the other one i gave you was incorrect http://www.python-forum.org/pythonforum/index.php -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
RE: PYTHON HTTP POST WORKING EXAMPLE NEEDED
hey didn't get the initial post/ just what is the original poster looking for?? -Original Message- From: python-list-bounces+bedouglas=earthlink@python.org [mailto:python-list-bounces+bedouglas=earthlink@python.org]on Behalf Of r Sent: Friday, January 23, 2009 5:03 PM To: python-list@python.org Subject: Re: PYTHON HTTP POST WORKING EXAMPLE NEEDED Whatever scumbag would rate a new python's users with one start for asking us for help is a complete low-life. Keep acting like that and Python will fade away into the darkness forever. dhaval, Don't let the few bullies here scare you away, try out the this link, the other one i gave you was incorrect http://www.python-forum.org/pythonforum/index.php -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: PYTHON HTTP POST
Does google accept POST?
Anyways, if you dont need to post files, you can use urlencode itself.
def encode_formdata(fields):
body = urllib.urlencode(dict(fields))
content_type = application/x-www-form-urlencoded
return content_type, body
If you need to post files too, then you will have to use multipart
data
def encode_multipart_formdata(fields, files):
fields is a sequence of (name, value) elements for regular form
fields.
files is a sequence of (name, filename, value) elements for data
to be uploaded as files
Return (content_type, body) ready for httplib.HTTP instance
BOUNDARY = '--ThIs_Is_tHe_bouNdaRY_$'
CRLF = '\r\n'
L = []
for (key, value) in fields:
L.append('--' + BOUNDARY)
L.append('Content-Disposition: form-data; name=%s' % key)
L.append('')
L.append(value)
for (key, filename, value) in files:
L.append('--' + BOUNDARY)
L.append('Content-Disposition: form-data; name=%s;
filename=%s' % (key, filename))
L.append('Content-Type: %s' % mimetypes.guess_type(filename)
[0] or 'application/octet-stream'
L.append('')
L.append(value)
L.append('--' + BOUNDARY + '--')
L.append('')
body = CRLF.join(L)
content_type = 'multipart/form-data; boundary=%s' % BOUNDARY
return content_type, body
Since POST files doesnt work with urllib, you might have to use
httplib - or go for very high level tools like twisted.
I here show an example with httplib.
def post(host, selector, fields, files):
if files:
content_type, body = encode_multipart_formdata(fields, files)
else:
content_type, body = encode_formdata(fields)
h = httplib.HTTPConnection(host)
#Spoof Mozilla
headers = {
'User-Agent': 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US;
rv:1.9.0.4) Gecko/2008102920 Firefox/3.0.4',
'Content-Type': content_type
}
h.request('POST', selector, body, headers)
res = h.getresponse()
return res.status, res.reason, res.read()
Please note that you can use multipart whether or not files are there,
but parsing multipart usually is slower.
Hope this helps.
lilanidha...@gmail.com wrote:
Hi,
I need one complete example of how to do a http post to any site.
I have tried making a POST to google but all I am returned with is a
405 error.
I don't want to use Pygoogle as I want to try and do this with other
sites.
I am also having problems inputing with the param
I have tried Mechanize. There are no problems with getting data only
posting.
headers = {'Content-Type': 'text/html; charset=ISO-8859-1',
... 'User-Agent':'Mozilla/4.0',
... 'Content-Length':'7'}
conn = httplib.HTTPConnection(www.google.com)
conn.request(POST,/search,params,headers)
r2 = conn.getresponse()
print r2.status, r2.reason
405 Method Not Allowed
Regards,
Dhaval
--
http://mail.python.org/mailman/listinfo/python-list
Re: PYTHON HTTP POST
On Jan 14, 2:21 pm, koranth...@gmail.com wrote:
Does google accept POST?
Anyways, if you dont need to post files, you can use urlencode itself.
def encode_formdata(fields):
body = urllib.urlencode(dict(fields))
content_type = application/x-www-form-urlencoded
return content_type, body
If you need to post files too, then you will have to use multipart
data
def encode_multipart_formdata(fields, files):
fields is a sequence of (name, value) elements for regular form
fields.
files is a sequence of (name, filename, value) elements for data
to be uploaded as files
Return (content_type, body) ready for httplib.HTTP instance
BOUNDARY = '--ThIs_Is_tHe_bouNdaRY_$'
CRLF = '\r\n'
L = []
for (key, value) in fields:
L.append('--' + BOUNDARY)
L.append('Content-Disposition: form-data; name=%s' % key)
L.append('')
L.append(value)
for (key, filename, value) in files:
L.append('--' + BOUNDARY)
L.append('Content-Disposition: form-data; name=%s;
filename=%s' % (key, filename))
L.append('Content-Type: %s' % mimetypes.guess_type(filename)
[0] or 'application/octet-stream'
L.append('')
L.append(value)
L.append('--' + BOUNDARY + '--')
L.append('')
body = CRLF.join(L)
content_type = 'multipart/form-data; boundary=%s' % BOUNDARY
return content_type, body
Since POST files doesnt work with urllib, you might have to use
httplib - or go for very high level tools like twisted.
I here show an example with httplib.
def post(host, selector, fields, files):
if files:
content_type, body = encode_multipart_formdata(fields, files)
else:
content_type, body = encode_formdata(fields)
h = httplib.HTTPConnection(host)
#Spoof Mozilla
headers = {
'User-Agent': 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US;
rv:1.9.0.4) Gecko/2008102920 Firefox/3.0.4',
'Content-Type': content_type
}
h.request('POST', selector, body, headers)
res = h.getresponse()
return res.status, res.reason, res.read()
Please note that you can use multipart whether or not files are there,
but parsing multipart usually is slower.
Hope this helps.
lilanidha...@gmail.com wrote:
Hi,
I need one complete example of how to do a http post to any site.
I have tried making a POST to google but all I am returned with is a
405 error.
I don't want to use Pygoogle as I want to try and do this with other
sites.
I am also having problems inputing with the param
I have tried Mechanize. There are no problems with getting data only
posting.
headers = {'Content-Type': 'text/html; charset=ISO-8859-1',
... 'User-Agent':'Mozilla/4.0',
... 'Content-Length':'7'}
conn = httplib.HTTPConnection(www.google.com)
conn.request(POST,/search,params,headers)
r2 = conn.getresponse()
print r2.status, r2.reason
405 Method Not Allowed
Regards,
Dhaval
oops - Forgot to mention that POSTing files mechanism is taken from a
recipe in active state -
http://code.activestate.com/recipes/146306/
--
http://mail.python.org/mailman/listinfo/python-list
Re: PYTHON HTTP POST
The action part of the field is not set to anything.
I need any site with working example that accepts POST.
On Jan 14, 2:21 pm, koranth...@gmail.com wrote:
Does google accept POST?
Anyways, if you dont need to post files, you can use urlencode itself.
def encode_formdata(fields):
body = urllib.urlencode(dict(fields))
content_type = application/x-www-form-urlencoded
return content_type, body
If you need to post files too, then you will have to use multipart
data
def encode_multipart_formdata(fields, files):
fields is a sequence of (name, value) elements for regular form
fields.
files is a sequence of (name, filename, value) elements for data
to be uploaded as files
Return (content_type, body) ready for httplib.HTTP instance
BOUNDARY = '--ThIs_Is_tHe_bouNdaRY_$'
CRLF = '\r\n'
L = []
for (key, value) in fields:
L.append('--' + BOUNDARY)
L.append('Content-Disposition: form-data; name=%s' % key)
L.append('')
L.append(value)
for (key, filename, value) in files:
L.append('--' + BOUNDARY)
L.append('Content-Disposition: form-data; name=%s;
filename=%s' % (key, filename))
L.append('Content-Type: %s' % mimetypes.guess_type(filename)
[0] or 'application/octet-stream'
L.append('')
L.append(value)
L.append('--' + BOUNDARY + '--')
L.append('')
body = CRLF.join(L)
content_type = 'multipart/form-data; boundary=%s' % BOUNDARY
return content_type, body
Since POST files doesnt work with urllib, you might have to use
httplib - or go for very high level tools like twisted.
I here show an example with httplib.
def post(host, selector, fields, files):
if files:
content_type, body = encode_multipart_formdata(fields, files)
else:
content_type, body = encode_formdata(fields)
h = httplib.HTTPConnection(host)
#Spoof Mozilla
headers = {
'User-Agent': 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US;
rv:1.9.0.4) Gecko/2008102920 Firefox/3.0.4',
'Content-Type': content_type
}
h.request('POST', selector, body, headers)
res = h.getresponse()
return res.status, res.reason, res.read()
Please note that you can use multipart whether or not files are there,
but parsing multipart usually is slower.
Hope this helps.
lilanidha...@gmail.com wrote:
Hi,
I need one complete example of how to do a http post to any site.
I have tried making a POST to google but all I am returned with is a
405 error.
I don't want to use Pygoogle as I want to try and do this with other
sites.
I am also having problems inputing with the param
I have tried Mechanize. There are no problems with getting data only
posting.
headers = {'Content-Type': 'text/html; charset=ISO-8859-1',
... 'User-Agent':'Mozilla/4.0',
... 'Content-Length':'7'}
conn = httplib.HTTPConnection(www.google.com)
conn.request(POST,/search,params,headers)
r2 = conn.getresponse()
print r2.status, r2.reason
405 Method Not Allowed
Regards,
Dhaval
--
http://mail.python.org/mailman/listinfo/python-list
Re: PYTHON HTTP POST WORKING EXAMPLE NEEDED
On Jan 14, 4:11 pm, dhaval lilanidha...@gmail.com wrote: Hi, Can someone please give me an example of a working python post? for example using the sitehttp://www.cookiemag.com/ Thanks in advance, Dhaval I have tried to look at the code at http://code.activestate.com/recipes/146306/ I can't make any sense out of it. -- http://mail.python.org/mailman/listinfo/python-list
Re: PYTHON HTTP POST WORKING EXAMPLE NEEDED
dhaval wrote: On Jan 14, 4:11 pm, dhaval lilanidha...@gmail.com wrote: Hi, Can someone please give me an example of a working python post? for example using the sitehttp://www.cookiemag.com/ Thanks in advance, Dhaval I have tried to look at the code at http://code.activestate.com/recipes/146306/ I can't make any sense out of it. - don't SHOUT. - don't open up new threads 2 hours after you asked the first time with the *same* question. - show code of your actual efforts. This is not the free code snippets producing system. - read this: http://www.catb.org/~esr/faqs/smart-questions.html Diez -- http://mail.python.org/mailman/listinfo/python-list
