Re: [PHP] PHP CURL JSON POST Firebug
On 04/09/2012 19:14, ioan...@btinternet.com wrote: On 04/09/2012 18:41, Matijn Woudt wrote: On Tue, Sep 4, 2012 at 7:35 PM, ioan...@btinternet.com ioan...@btinternet.com wrote: I am hoping someone can spot what is missing here. I am getting null result from curl-ing a page with json post variables. I try this url in my Firefox browser - http://www.targetsite.com/search.php#somevar.someothervar (#somevar.someothervar are irrelevant, I think, as I get the curl variables from Firebug below.) In Firebug, this shows: POST http://www.targetsite.com/ajax/search.xml In Firebug, below this link are tabs for: Headers, Post, Response, XML and Cookies. Post tab shows like: JSON VAR1 1 VAR2 2012-09-12 VAR3 null CACHED [] OPTIONSnull To prove there is output, the Firebug Response tab shows: ?xml version=1.0 encoding=utf-8? JSON various JSON encoded stuff /JSON The above is what I am trying to curl. My php code: $ch = curl_init(); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch, CURLOPT_POST, 1); //target page from Firebug above: curl_setopt($ch, CURLOPT_URL, http://www.targetsite.com/ajax/search.xml;); //I was not sure how to represent CACHED [], so set it to null try CACHED=array() $data = array( VAR1 = 1, VAR2 = 2012-09-12, VAR3 = null, CACHED=null, OPTIONS=null, ); curl_setopt($ch, CURLOPT_POSTFIELDS, $data); //make the request $result = curl_exec($ch); //this returns null Any ideas where to go with this? Maybe I need to include the Cookies? I use the above php and curl functions normally so it's all installed on the server. John It might be that the site is using sessions/cookies. Have a look at the header data with firebug. Not sure if that's the problem, to find out what's really going on, call echo curl_error($ch); after curl_exec to find out what went wrong exactly. If you still don't know how to proceed, paste the result of the curl_error call in your reply. - Matijn I added the cookies to the post array. I changed php array to CACHED=array() for the JSON CACHED:[], and corrected php's null to NULL. It is not returning any error. The browser was showing 'resource not present' before I added the cookies to the post array, now it just returns null $result. Looks like I am transcribing something incorrectly. John I eventually sorted this out. Solution involved: POST params needed to be json_encoded $params=json_encode(array( name = value )); Thanks to http://www.lornajane.net/posts/2011/posting-json-data-with-php-curl curl_setopt($ch, CURLOPT_CUSTOMREQUEST, POST); curl_setopt($ch, CURLOPT_POSTFIELDS, $post); Also, included headers as array and set application type as json: curl_setopt($ch, CURLOPT_HTTPHEADER, array( 'Content-Type: application/json', 'Content-Length: ' . strlen($post)) ); Set encoding to auto-detect: curl_setopt( $ch, CURLOPT_ENCODING, ); John -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] PHP CURL JSON POST Firebug
I am hoping someone can spot what is missing here. I am getting null result from curl-ing a page with json post variables. I try this url in my Firefox browser - http://www.targetsite.com/search.php#somevar.someothervar (#somevar.someothervar are irrelevant, I think, as I get the curl variables from Firebug below.) In Firebug, this shows: POST http://www.targetsite.com/ajax/search.xml In Firebug, below this link are tabs for: Headers, Post, Response, XML and Cookies. Post tab shows like: JSON VAR1 1 VAR2 2012-09-12 VAR3 null CACHED [] OPTIONSnull To prove there is output, the Firebug Response tab shows: ?xml version=1.0 encoding=utf-8? JSON various JSON encoded stuff /JSON The above is what I am trying to curl. My php code: $ch = curl_init(); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch, CURLOPT_POST, 1); //target page from Firebug above: curl_setopt($ch, CURLOPT_URL, http://www.targetsite.com/ajax/search.xml;); //I was not sure how to represent CACHED [], so set it to null $data = array( VAR1 = 1, VAR2 = 2012-09-12, VAR3 = null, CACHED=null, OPTIONS=null, ); curl_setopt($ch, CURLOPT_POSTFIELDS, $data); //make the request $result = curl_exec($ch); //this returns null Any ideas where to go with this? Maybe I need to include the Cookies? I use the above php and curl functions normally so it's all installed on the server. John -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] PHP CURL JSON POST Firebug
On Tue, Sep 4, 2012 at 7:35 PM, ioan...@btinternet.com ioan...@btinternet.com wrote: I am hoping someone can spot what is missing here. I am getting null result from curl-ing a page with json post variables. I try this url in my Firefox browser - http://www.targetsite.com/search.php#somevar.someothervar (#somevar.someothervar are irrelevant, I think, as I get the curl variables from Firebug below.) In Firebug, this shows: POST http://www.targetsite.com/ajax/search.xml In Firebug, below this link are tabs for: Headers, Post, Response, XML and Cookies. Post tab shows like: JSON VAR1 1 VAR2 2012-09-12 VAR3 null CACHED [] OPTIONSnull To prove there is output, the Firebug Response tab shows: ?xml version=1.0 encoding=utf-8? JSON various JSON encoded stuff /JSON The above is what I am trying to curl. My php code: $ch = curl_init(); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch, CURLOPT_POST, 1); //target page from Firebug above: curl_setopt($ch, CURLOPT_URL, http://www.targetsite.com/ajax/search.xml;); //I was not sure how to represent CACHED [], so set it to null try CACHED=array() $data = array( VAR1 = 1, VAR2 = 2012-09-12, VAR3 = null, CACHED=null, OPTIONS=null, ); curl_setopt($ch, CURLOPT_POSTFIELDS, $data); //make the request $result = curl_exec($ch); //this returns null Any ideas where to go with this? Maybe I need to include the Cookies? I use the above php and curl functions normally so it's all installed on the server. John It might be that the site is using sessions/cookies. Have a look at the header data with firebug. Not sure if that's the problem, to find out what's really going on, call echo curl_error($ch); after curl_exec to find out what went wrong exactly. If you still don't know how to proceed, paste the result of the curl_error call in your reply. - Matijn -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] PHP CURL JSON POST Firebug
On 04/09/2012 18:41, Matijn Woudt wrote: On Tue, Sep 4, 2012 at 7:35 PM, ioan...@btinternet.com ioan...@btinternet.com wrote: I am hoping someone can spot what is missing here. I am getting null result from curl-ing a page with json post variables. I try this url in my Firefox browser - http://www.targetsite.com/search.php#somevar.someothervar (#somevar.someothervar are irrelevant, I think, as I get the curl variables from Firebug below.) In Firebug, this shows: POST http://www.targetsite.com/ajax/search.xml In Firebug, below this link are tabs for: Headers, Post, Response, XML and Cookies. Post tab shows like: JSON VAR1 1 VAR2 2012-09-12 VAR3 null CACHED [] OPTIONSnull To prove there is output, the Firebug Response tab shows: ?xml version=1.0 encoding=utf-8? JSON various JSON encoded stuff /JSON The above is what I am trying to curl. My php code: $ch = curl_init(); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch, CURLOPT_POST, 1); //target page from Firebug above: curl_setopt($ch, CURLOPT_URL, http://www.targetsite.com/ajax/search.xml;); //I was not sure how to represent CACHED [], so set it to null try CACHED=array() $data = array( VAR1 = 1, VAR2 = 2012-09-12, VAR3 = null, CACHED=null, OPTIONS=null, ); curl_setopt($ch, CURLOPT_POSTFIELDS, $data); //make the request $result = curl_exec($ch); //this returns null Any ideas where to go with this? Maybe I need to include the Cookies? I use the above php and curl functions normally so it's all installed on the server. John It might be that the site is using sessions/cookies. Have a look at the header data with firebug. Not sure if that's the problem, to find out what's really going on, call echo curl_error($ch); after curl_exec to find out what went wrong exactly. If you still don't know how to proceed, paste the result of the curl_error call in your reply. - Matijn I added the cookies to the post array. I changed php array to CACHED=array() for the JSON CACHED:[], and corrected php's null to NULL. It is not returning any error. The browser was showing 'resource not present' before I added the cookies to the post array, now it just returns null $result. Looks like I am transcribing something incorrectly. John -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] I need help with PHP, cURL, and POST
On Tue, January 2, 2007 8:56 pm, Charley wrote: What do you mean about the submit input? I thought that was just a text link to cause the form to be submitted when clicked. I didn't know that it sent anything to the server. (As you can tell, I don't know anything about asp except what it stands for.) input type=submit / This line in HTML, with variations, *WILL* send another submit=Submit as part of the HTTP request. If you don't send that part, ASP scripts will puke on your pretty blue suede shoes, more often than not, in my experience. Never mind that Netscape 2 didn't send this field when you hit Enter instead of actually clicking the button for a one-button form (default action: submit). MS never cared about interoperability then, and they don't now, and I don't expect them to ever care in the future, no matter how many times they claim to embrace and extend (or whatever their slogan is this week). The paymentidfilter is a legitimate field, and works in the form and in the GET version of the cURL stuff. Okay -- It just wasn't in the FORM, so wanted to check. It seems that e-gold has as its error response to send the login page for whatever one is asking for. This is not very helpful (at least to me). Oh, but it's *VERY* helpful. It tells you that you are not proving to the server that you are a legitimate logged-in user. Which, in turn, means that somehow you need to do just that -- Fool their server into accepting and maintaining your login credentials. What this usually entails is something like this: $curl = curl_init(); //login: curl_setopt(CURLOPT_URL, .../login.asp); curl_setopt(CURLOPT_POSTDATA, username=foopassword=bar); curl_setopt(CURLOPT_COOKIEJAR, /full/path/to/php/writable/cookie/storage/file.txt); curl_setopt(CURLOPT_COOKIEFILE, /full/path/to/php/writable/cookie/storage/file.txt); //get data we actually want //the script you currently have gets pasted in here I'll see if I can get this LiveHTTPheaders thing for firefox, since firefox is my main browser. From what I read, the cookie jar and file options take the name of the same file as their arguments. Is there any path information that is required? Can I just use something like mycookies.txt or must it have a specific extension? The only restriction is that it MUST be readable/writable by the User php runs as, so that PHP can write down the cookies it gets, and spit them back to prove that it is still a valid user. You also cannot use the COOKIE(JAR|FILE) with the CURLOPT_HEADER option that lets you see the other headers. :-( If you need the other headers, you have to manage the cookies on your own instead of letting curl do it for you. This is not all that tricky, really, once you look at what a Cookie actually looks like in the headers. -- Some people have a gift link here. Know what I want? I want you to buy a CD from some starving artist. http://cdbaby.com/browse/from/lynch Yeah, I get a buck. So? -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] I need help with PHP, cURL, and POST
I am an experienced programmer who is just learning php and curl. I have tried for several days to figure out how to use curl with POST to get history information from e-gold. The following script, which I guess uses GET works when have appropriate stuff in them. But no matter what I have tried, and whose examples I have followed I cannot get this to work with CURLOPT_POST set to true with what I think is the appropriate information supplied. I am testing this from XAMPP 1.5.5 on a Windows XP system, and the SSL stuff and everything I think I need is present and seems to be working fine. The commented out form also works, so I guess e-gold will accept POSTed data. I just do not know how to get curl to send it for me or what options I need to make it work. Would someone be interested in showing me exactly what I need to modify this script to work with POST? ?php session_start(); /* form action=https://www.e-gold.com/acct/historycsv.asp; method=post input type=hidden name=AccountID value=XX input type=hidden name=PassPhrase value=YYY input type=hidden name=startmonth value=12 input type=hidden name=startday value=1 input type=hidden name=startyear value=2006 input type=hidden name=endmonth value=12 input type=hidden name=endday value=31 input type=hidden name=endyear value=2006 input type=hidden name=paymentsreceived value=1 input type=hidden name=fees value=1 input type=submit value=Submit */ $pf = AccountID=XX; $pf .= PassPhrase=YYY; $pf .= startmonth=12; $pf .= startday=1; $pf .= startyear=2006; $pf .= endmonth=12; $pf .= endday=31; $pf .= endyear=2006; $pf .= paymentsreceived=1; $pf .= fees=1; $pf .= paymentidfilter=; $ch = curl_init(); // Follow any Location headers curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); $url='https://www.e-gold.com/acct/historycsv.asp?' . $pf; curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE); curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0); curl_setopt($ch, CURLOPT_HEADER, FALSE); $data=curl_exec($ch); $info=curl_getinfo($ch); echo pcurlerror= . curl_error($ch); curl_close($ch); echo pdata=; var_dump($data); echo pinfo=; var_dump($info); ?
Re: [PHP] I need help with PHP, cURL, and POST
Charley wrote: I am an experienced programmer who is just learning php and curl. ... Would someone be interested in showing me exactly what I need to modify this script to work with POST? the interest is 10%/week ;-) ... read on ... ?php session_start(); /* form action=https://www.e-gold.com/acct/historycsv.asp; method=post input type=hidden name=AccountID value=XX input type=hidden name=PassPhrase value=YYY input type=hidden name=startmonth value=12 input type=hidden name=startday value=1 input type=hidden name=startyear value=2006 input type=hidden name=endmonth value=12 input type=hidden name=endday value=31 input type=hidden name=endyear value=2006 input type=hidden name=paymentsreceived value=1 input type=hidden name=fees value=1 input type=submit value=Submit */ $pf = AccountID=XX; $pf .= PassPhrase=YYY; $pf .= startmonth=12; $pf .= startday=1; $pf .= startyear=2006; $pf .= endmonth=12; $pf .= endday=31; $pf .= endyear=2006; $pf .= paymentsreceived=1; $pf .= fees=1; $pf .= paymentidfilter=; you don't want/need the GET query in $pf ... instead something like this needs to be used (AFAIK): curl_setopt($ch, CURLOPT_POSTFIELDS, array( 'AccountID' = 'X', 'PassPhrase' = 'XX', /* etc, etc */ )); refer to http://php.net/curl for more info. this assumes that e-gold does accept POST as a request method (which you test form seems to indicate it does) $ch = curl_init(); // Follow any Location headers curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); $url='https://www.e-gold.com/acct/historycsv.asp?' . $pf; curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE); curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0); curl_setopt($ch, CURLOPT_HEADER, FALSE); $data=curl_exec($ch); $info=curl_getinfo($ch); echo pcurlerror= . curl_error($ch); curl_close($ch); echo pdata=; var_dump($data); echo pinfo=; var_dump($info); ? -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP] I need help with PHP, cURL, and POST
Have you looked at this tutorial - http://devzone.zend.com/node/view/id/1081 It has an example specifically for how to use curl with POST Respectfully, Ligaya Turmelle -Original Message- From: Charley [mailto:[EMAIL PROTECTED] Sent: Wednesday, January 03, 2007 1:24 AM To: php-general@lists.php.net Subject: [PHP] I need help with PHP, cURL, and POST I am an experienced programmer who is just learning php and curl. I have tried for several days to figure out how to use curl with POST to get history information from e-gold. The following script, which I guess uses GET works when have appropriate stuff in them. But no matter what I have tried, and whose examples I have followed I cannot get this to work with CURLOPT_POST set to true with what I think is the appropriate information supplied. I am testing this from XAMPP 1.5.5 on a Windows XP system, and the SSL stuff and everything I think I need is present and seems to be working fine. The commented out form also works, so I guess e-gold will accept POSTed data. I just do not know how to get curl to send it for me or what options I need to make it work. Would someone be interested in showing me exactly what I need to modify this script to work with POST? ?php session_start(); /* form action=https://www.e-gold.com/acct/historycsv.asp; method=post input type=hidden name=AccountID value=XX input type=hidden name=PassPhrase value=YYY input type=hidden name=startmonth value=12 input type=hidden name=startday value=1 input type=hidden name=startyear value=2006 input type=hidden name=endmonth value=12 input type=hidden name=endday value=31 input type=hidden name=endyear value=2006 input type=hidden name=paymentsreceived value=1 input type=hidden name=fees value=1 input type=submit value=Submit */ $pf = AccountID=XX; $pf .= PassPhrase=YYY; $pf .= startmonth=12; $pf .= startday=1; $pf .= startyear=2006; $pf .= endmonth=12; $pf .= endday=31; $pf .= endyear=2006; $pf .= paymentsreceived=1; $pf .= fees=1; $pf .= paymentidfilter=; $ch = curl_init(); // Follow any Location headers curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); $url='https://www.e-gold.com/acct/historycsv.asp?' . $pf; curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE); curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0); curl_setopt($ch, CURLOPT_HEADER, FALSE); $data=curl_exec($ch); $info=curl_getinfo($ch); echo pcurlerror= . curl_error($ch); curl_close($ch); echo pdata=; var_dump($data); echo pinfo=; var_dump($info); ? -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] I need help with PHP, cURL, and POST
To Ligaya and Jochm, Thanks for your answers. I have read and tried what several tutorials say to do. I have used both arrays and query strings to specify the post fields. I had not seen the tutorial Ligaya mentioned, but it was not essentially different from the others I have seen. And the PHP cURL documentation assumes that I know a lot more about what each option means than I do. I have been reading and re-reading that for days now. I have the feeling that there is a CURLOPT_something or other that I should be using and am not, but I have no idea what it could be. I thought it might have to do with cookies or verification or something, but I think I have tried all possible relevant combinations of those things. I guess I will let it go for a while -- I was hoping someone would tell me that they had thus and such problem and did this and that to fix it and that that would work for me. (OK, so I'm an optimist (optomist??)) Thanks again, guys.
[PHP] I need help with PHP, cURL, and POST
I apologize for the previous message. This one should wrap better. To Ligaya and Jochm, Thanks for your answers. I have read and tried what several tutorials say to do. I have used both arrays and query strings to specify the post fields. I had not seen the tutorial Ligaya mentioned, but it was not essentially different from the others I have seen. And the PHP cURL documentation assumes that I know a lot more about what each option means than I do. I have been reading and re-reading that for days now. I have the feeling that there is a CURLOPT_something or other that I should be using and am not, but I have no idea what it could be. I thought it might have to do with cookies or verification or something, but I think I have tried all possible relevant combinations of those things. I guess I will let it go for a while -- I was hoping someone would tell me that they had thus and such problem and did this and that to fix it and that that would work for me. (OK, so I'm an optimist (optomist??)) Thanks again, guys. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] I need help with PHP, cURL, and POST
On Tue, January 2, 2007 9:23 am, Charley wrote: input type=hidden name=AccountID value=XX ... input type=submit value=Submit */ $pf = AccountID=XX; ... $pf .= paymentidfilter=; You are missing the Submit input, and ASP being ASP, it probably was programmed to expect it, and will puke without it. You also have added a paymentidfilter parameter that was not in the original. You may not be allowed to have that in the POST, perhaps. $ch = curl_init(); ... echo pcurlerror= . curl_error($ch); Based on the rest of your excellent post, you probably have no error output here, but to be pedantic... Are there any error messages here? You may also want to turn off the FOLLOWLOCATION and dump out whatever responses as the come, and then build up another query to follow the re-directs by hand. This will sometimes lead you to find out that there are, for example, cookies flying back and forth that you need to track (CURLOPT_COOKIEJAR and CURLOPT_COOKIEFILE) or something similar. You may also want to provide some extra headers to make curl look more like a real browser to fool any anti-web-scraping features in their ASP script. Compare output with Firefox LiveHTTPHeaders (?) and what curl is giving you to find clues/differences that might indicate what to try next. Keep in mind that your goal is to make your curl script indistinguishable to e-gold from a real user. -- Some people have a gift link here. Know what I want? I want you to buy a CD from some starving artist. http://cdbaby.com/browse/from/lynch Yeah, I get a buck. So? -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] I need help with PHP, cURL, and POST
Richard, Thanks for your answer. What do you mean about the submit input? I thought that was just a text link to cause the form to be submitted when clicked. I didn't know that it sent anything to the server. (As you can tell, I don't know anything about asp except what it stands for.) The paymentidfilter is a legitimate field, and works in the form and in the GET version of the cURL stuff. You're right that I get no curl error, and the curl info didn't seem to have anything interesting in it. It seems that e-gold has as its error response to send the login page for whatever one is asking for. This is not very helpful (at least to me). I'll see if I can get this LiveHTTPheaders thing for firefox, since firefox is my main browser. From what I read, the cookie jar and file options take the name of the same file as their arguments. Is there any path information that is required? Can I just use something like mycookies.txt or must it have a specific extension? Thanks again. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] I need help with PHP, cURL, and POST
It works! All I did was comment out the curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1) and set up the post and post fields options. I used the query string format. I may try the array later, but since this works, I'm going to use it for now. Thanks everybody! -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] Curl and post
I need to send a post array to another page. How do I do this using CURL? -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] cURL and POST
HI, I'm using the cURL binary to perform a HTTP POST to a web page. I am sending XML data to this web page. The issue I've come up against if the size of the data that I'm trying to post and it looks like cURL is crapping out. This is the command line that I'm currently using: /usr/local/bin/curl -m 600 -d $strXML http://myurl.com/webpage.php -L -s $strXML - variable containing the XML data. I run this command using exec. The return code I receive is: 127. This means a fatal error has occured with cURL but that's all I know. I have read the detail about using the -F option. I do not want to POST the file as such, but rather the DATA in the file is I create a file of the XML. The manpage for cURL states: -F : To just get the content part from a file, prefix the file name with the letter . The difference between and is then that makes a file get attached in the post as a file upload, while the makes a text field and just get the contents for that text field from a file. So I've tried going: /usr/local/bin/curl -m 600 -F /tmp/tempfile.xml http://myurl.com/webpage.php -L -s /tmp/tempfile.xml - contains the XML data. This isn't working.I'm getting a return code of 2 meaning: Failed to initialize. I'm at a loss. I need to post this large amount of XML data to a remote site and don't know how to get it there. Any ideas? Thanks, Sam -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
