RES: [PHP] cURL issues posting to an end point
Bastien:
curl_setopt($ch, CURLOPT_HTTPHEADER,
array(
'User-Agent: My Program',
'Accept-Encoding: gzip, deflate',
'POST ' . $sPost . ' HTTP/1.1',
'Host: ' . $sHost,
'Content-type: application/soap+xml; charset="utf-8"',
'Content-Length: ' . strlen($xml_builder)
) --- This is missing
);
Alejandro M.S
-Mensagem original-
De: Bastien Koert [mailto:phps...@gmail.com]
Enviada em: quinta-feira, 4 de outubro de 2012 15:49
Para: Alejandro Michelin Salomon
Cc: PHP-General
Assunto: Re: [PHP] cURL issues posting to an end point
On Thu, Oct 4, 2012 at 1:35 PM, Alejandro Michelin Salomon
wrote:
> Bastien:
>
> -Mensagem original-
> De: Bastien Koert [mailto:phps...@gmail.com] Enviada em: quinta-feira,
> 4 de outubro de 2012 11:54
> Para: PHP-General
> Assunto: [PHP] cURL issues posting to an end point
>
> Hi All,
>
> I have a page that receives third party data into my app, xml data via
> https post. This works fine and I receive the data as expected. The
> issue I am facing is around posting XML data back as a synchronous
> response to the post I receive. I am using the following code:
>
> function sendXMLConfirmation($data)
> {
> /*
>* XML Sender/Client.
>*/
> // Get our XML. You can declare it here or even load a file.
>
>
> $xml_builder = '
>
>
>
>
> 25412
>
>
>
>
> transactType="response">
>
> '.$hash.'
>
> '.date("Y-m-d H:m ").'
>
>
> 200
>
> Success
>
> CANDIDATE Data transfer was a success
>
>
>
> packetType="response">
>
> 1
>
SET
>
> Manifest Data
>
>
>
>
>
>
>
>
>
>
>
>
>
> ';
>
>
> // We send XML via CURL using POST with a http header of text/xml.
> $ch = curl_init();
>
> $url =
> 'https://staging.somesite.com/Sprocessor/DispatchMessage.asmx';
>
> curl_setopt($ch, CURLOPT_URL, $url);
> curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); // for https
> curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
> curl_setopt($ch, CURLOPT_TIMEOUT, 4);
> curl_setopt($ch, CURLOPT_POSTFIELDS, $xml_builder);
>
> - curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type:
text/xml'));
> - curl_setopt($ch, CURLOPT_HTTPHEADER, array('Connection: close'));
>
> --- NEW
> $sPost = '/Sprocessor/DispatchMessage.asmx';
> $sHost = ' https://staging.somesite.com';
>
> curl_setopt($ch, CURLOPT_HTTPHEADER,
> array(
> 'User-Agent: My Program',
> 'Accept-Encoding: gzip, deflate',
> 'POST ' . $sPost . ' HTTP/1.1',
> 'Host: ' . $sHost,
> 'Content-type: application/soap+xml; charset="utf-8"',
> 'Content-Length: ' . strlen($xml_builder) );
>
> END NEW -
> //Execute the request and also time the transaction ( optional
> )
>
> $start = array_sum(explode(' ', microtime()));
>
> $result = curl_exec($ch);
>
> curl_close($ch);
> // Print CURL result.
> echo $ch_result;
> }
>
>
> The endpoint recipient says they are not receiving the data and I am
> at a loss to figure out why. Any insight would be appreciated.
>
> Thanks,
>
> --
>
> Bastien
>
> Cat, the other other white meat
>
> --
> PHP General Mailing List (http://www.php.net/) To unsubscribe, visit
Re: [PHP] cURL issues posting to an end point
On Thu, Oct 4, 2012 at 1:35 PM, Alejandro Michelin Salomon
wrote:
> Bastien:
>
> -Mensagem original-
> De: Bastien Koert [mailto:phps...@gmail.com]
> Enviada em: quinta-feira, 4 de outubro de 2012 11:54
> Para: PHP-General
> Assunto: [PHP] cURL issues posting to an end point
>
> Hi All,
>
> I have a page that receives third party data into my app, xml data via https
> post. This works fine and I receive the data as expected. The issue I am
> facing is around posting XML data back as a synchronous response to the post
> I receive. I am using the following code:
>
> function sendXMLConfirmation($data)
> {
> /*
>* XML Sender/Client.
>*/
> // Get our XML. You can declare it here or even load a file.
>
>
> $xml_builder = '
>
>
>
>
> 25412
>
>
>
>
> transactType="response">
>
> '.$hash.'
> '.date("Y-m-d H:m
> ").'
>
> 200
>
> Success
>
> CANDIDATE Data transfer was a success
>
>
>
> packetType="response">
>
> 1
> SET
> Manifest
> Data
>
>
>
>
>
>
>
>
>
>
>
>
> ';
>
>
> // We send XML via CURL using POST with a http header of text/xml.
> $ch = curl_init();
>
> $url =
> 'https://staging.somesite.com/Sprocessor/DispatchMessage.asmx';
>
> curl_setopt($ch, CURLOPT_URL, $url);
> curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); // for https
> curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
> curl_setopt($ch, CURLOPT_TIMEOUT, 4);
> curl_setopt($ch, CURLOPT_POSTFIELDS, $xml_builder);
>
> - curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type: text/xml'));
> - curl_setopt($ch, CURLOPT_HTTPHEADER, array('Connection: close'));
>
> --- NEW
> $sPost = '/Sprocessor/DispatchMessage.asmx';
> $sHost = ' https://staging.somesite.com';
>
> curl_setopt($ch, CURLOPT_HTTPHEADER,
> array(
> 'User-Agent: My Program',
> 'Accept-Encoding: gzip, deflate',
> 'POST ' . $sPost . ' HTTP/1.1',
> 'Host: ' . $sHost,
> 'Content-type: application/soap+xml; charset="utf-8"',
> 'Content-Length: ' . strlen($xml_builder)
> );
>
> END NEW -
> //Execute the request and also time the transaction ( optional )
>
> $start = array_sum(explode(' ', microtime()));
>
> $result = curl_exec($ch);
>
> curl_close($ch);
> // Print CURL result.
> echo $ch_result;
> }
>
>
> The endpoint recipient says they are not receiving the data and I am at a
> loss to figure out why. Any insight would be appreciated.
>
> Thanks,
>
> --
>
> Bastien
>
> Cat, the other other white meat
>
> --
> PHP General Mailing List (http://www.php.net/) To unsubscribe, visit:
> http://www.php.net/unsub.php
>
>
Thanks, Alejandro,
So much closer that before, however I am getting an error on this.
"Unexpected ';' on line 151 which is the closing array );
curl_setopt($ch, CURLOPT_HTTPHEADER,
array(
'User-Agent: My Program',
'Accept-Encoding: gzip, deflate',
'POST ' . $sPost . ' HTTP/1.1',
'Host: ' . $sHost,
'Content-type: application/soap+xml; charset="utf-8"',
'Content-Length: ' . strlen($xml_builder)
);
--
Bastien
Cat, the other other white meat
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
RES: [PHP] cURL issues posting to an end point
Bastien:
-Mensagem original-
De: Bastien Koert [mailto:phps...@gmail.com]
Enviada em: quinta-feira, 4 de outubro de 2012 11:54
Para: PHP-General
Assunto: [PHP] cURL issues posting to an end point
Hi All,
I have a page that receives third party data into my app, xml data via https
post. This works fine and I receive the data as expected. The issue I am
facing is around posting XML data back as a synchronous response to the post
I receive. I am using the following code:
function sendXMLConfirmation($data)
{
/*
* XML Sender/Client.
*/
// Get our XML. You can declare it here or even load a file.
$xml_builder = '
25412
'.$hash.'
'.date("Y-m-d H:m
").'
200
Success
CANDIDATE Data transfer was a success
1
SET
Manifest
Data
';
// We send XML via CURL using POST with a http header of text/xml.
$ch = curl_init();
$url =
'https://staging.somesite.com/Sprocessor/DispatchMessage.asmx';
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); // for https
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_TIMEOUT, 4);
curl_setopt($ch, CURLOPT_POSTFIELDS, $xml_builder);
- curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type: text/xml'));
- curl_setopt($ch, CURLOPT_HTTPHEADER, array('Connection: close'));
--- NEW
$sPost = '/Sprocessor/DispatchMessage.asmx';
$sHost = ' https://staging.somesite.com';
curl_setopt($ch, CURLOPT_HTTPHEADER,
array(
'User-Agent: My Program',
'Accept-Encoding: gzip, deflate',
'POST ' . $sPost . ' HTTP/1.1',
'Host: ' . $sHost,
'Content-type: application/soap+xml; charset="utf-8"',
'Content-Length: ' . strlen($xml_builder)
);
END NEW -
//Execute the request and also time the transaction ( optional )
$start = array_sum(explode(' ', microtime()));
$result = curl_exec($ch);
curl_close($ch);
// Print CURL result.
echo $ch_result;
}
The endpoint recipient says they are not receiving the data and I am at a
loss to figure out why. Any insight would be appreciated.
Thanks,
--
Bastien
Cat, the other other white meat
--
PHP General Mailing List (http://www.php.net/) To unsubscribe, visit:
http://www.php.net/unsub.php
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] cURL issues posting to an end point
Hi All,
I have a page that receives third party data into my app, xml data via
https post. This works fine and I receive the data as expected. The
issue I am facing is around posting XML data back as a synchronous
response to the post I receive. I am using the following code:
function sendXMLConfirmation($data)
{
/*
* XML Sender/Client.
*/
// Get our XML. You can declare it here or even load a file.
$xml_builder = '
25412
'.$hash.'
'.date("Y-m-d H:m
").'
200
Success
CANDIDATE Data transfer was a success
1
SET
Manifest
Data
';
// We send XML via CURL using POST with a http header of text/xml.
$ch = curl_init();
$url = 'https://staging.somesite.com/Sprocessor/DispatchMessage.asmx';
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); // for https
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_TIMEOUT, 4);
curl_setopt($ch, CURLOPT_POSTFIELDS, $xml_builder);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type: text/xml'));
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Connection: close'));
//Execute the request and also time the transaction ( optional )
$start = array_sum(explode(' ', microtime()));
$result = curl_exec($ch);
curl_close($ch);
// Print CURL result.
echo $ch_result;
}
The endpoint recipient says they are not receiving the data and I am
at a loss to figure out why. Any insight would be appreciated.
Thanks,
--
Bastien
Cat, the other other white meat
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] CURL vs Exif_imagetype()
Hi, I host a flash painting tool on my site, and wanted to add ability for the users to add a background image from a given url, but I have to make sure the url they add truely refers to a real jpg file, and not something else. I found a methoed with exif_imagetype() that worked except my webhost has disabled the allow_url_fopen due to security issues, instead they recommented me to use CURL and said that the exif_imagetype() is not a modern way to use php today. (what do I know)? Question: Is it possible for me to use CURL to verify the authenticity of a jpg file? ...and if so, how? Regards from Rango` -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] PHP CURL JSON POST Firebug
On 04/09/2012 19:14, ioan...@btinternet.com wrote: On 04/09/2012 18:41, Matijn Woudt wrote: On Tue, Sep 4, 2012 at 7:35 PM, ioan...@btinternet.com wrote: I am hoping someone can spot what is missing here. I am getting null result from curl-ing a page with json post variables. I try this url in my Firefox browser - http://www.targetsite.com/search.php#somevar.someothervar (#somevar.someothervar are irrelevant, I think, as I get the curl variables from Firebug below.) In Firebug, this shows: POST http://www.targetsite.com/ajax/search.xml In Firebug, below this link are tabs for: Headers, Post, Response, XML and Cookies. Post tab shows like: JSON VAR1 1 VAR2 "2012-09-12" VAR3 null CACHED [] OPTIONSnull To prove there is output, the Firebug Response tab shows: various JSON encoded stuff The above is what I am trying to curl. My php code: $ch = curl_init(); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch, CURLOPT_POST, 1); //target page from Firebug above: curl_setopt($ch, CURLOPT_URL, "http://www.targetsite.com/ajax/search.xml";); //I was not sure how to represent CACHED [], so set it to null try "CACHED"=>array() $data = array( "VAR1" => 1, "VAR2" => "2012-09-12", "VAR3" => null, "CACHED"=>null, "OPTIONS"=>null, ); curl_setopt($ch, CURLOPT_POSTFIELDS, $data); //make the request $result = curl_exec($ch); //this returns null Any ideas where to go with this? Maybe I need to include the Cookies? I use the above php and curl functions normally so it's all installed on the server. John It might be that the site is using sessions/cookies. Have a look at the header data with firebug. Not sure if that's the problem, to find out what's really going on, call echo curl_error($ch); after curl_exec to find out what went wrong exactly. If you still don't know how to proceed, paste the result of the curl_error call in your reply. - Matijn I added the cookies to the post array. I changed php array to "CACHED"=>array() for the JSON "CACHED":[], and corrected php's null to NULL. It is not returning any error. The browser was showing 'resource not present' before I added the cookies to the post array, now it just returns null $result. Looks like I am transcribing something incorrectly. John I eventually sorted this out. Solution involved: POST params needed to be json_encoded $params=json_encode(array( "name" => "value" )); Thanks to http://www.lornajane.net/posts/2011/posting-json-data-with-php-curl curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST"); curl_setopt($ch, CURLOPT_POSTFIELDS, $post); Also, included headers as array and set application type as json: curl_setopt($ch, CURLOPT_HTTPHEADER, array( 'Content-Type: application/json', 'Content-Length: ' . strlen($post)) ); Set encoding to auto-detect: curl_setopt( $ch, CURLOPT_ENCODING, ""); John -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] PHP CURL JSON POST Firebug
On 04/09/2012 18:41, Matijn Woudt wrote: On Tue, Sep 4, 2012 at 7:35 PM, ioan...@btinternet.com wrote: I am hoping someone can spot what is missing here. I am getting null result from curl-ing a page with json post variables. I try this url in my Firefox browser - http://www.targetsite.com/search.php#somevar.someothervar (#somevar.someothervar are irrelevant, I think, as I get the curl variables from Firebug below.) In Firebug, this shows: POST http://www.targetsite.com/ajax/search.xml In Firebug, below this link are tabs for: Headers, Post, Response, XML and Cookies. Post tab shows like: JSON VAR1 1 VAR2 "2012-09-12" VAR3 null CACHED [] OPTIONSnull To prove there is output, the Firebug Response tab shows: various JSON encoded stuff The above is what I am trying to curl. My php code: $ch = curl_init(); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch, CURLOPT_POST, 1); //target page from Firebug above: curl_setopt($ch, CURLOPT_URL, "http://www.targetsite.com/ajax/search.xml";); //I was not sure how to represent CACHED [], so set it to null try "CACHED"=>array() $data = array( "VAR1" => 1, "VAR2" => "2012-09-12", "VAR3" => null, "CACHED"=>null, "OPTIONS"=>null, ); curl_setopt($ch, CURLOPT_POSTFIELDS, $data); //make the request $result = curl_exec($ch); //this returns null Any ideas where to go with this? Maybe I need to include the Cookies? I use the above php and curl functions normally so it's all installed on the server. John It might be that the site is using sessions/cookies. Have a look at the header data with firebug. Not sure if that's the problem, to find out what's really going on, call echo curl_error($ch); after curl_exec to find out what went wrong exactly. If you still don't know how to proceed, paste the result of the curl_error call in your reply. - Matijn I added the cookies to the post array. I changed php array to "CACHED"=>array() for the JSON "CACHED":[], and corrected php's null to NULL. It is not returning any error. The browser was showing 'resource not present' before I added the cookies to the post array, now it just returns null $result. Looks like I am transcribing something incorrectly. John -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] PHP CURL JSON POST Firebug
On Tue, Sep 4, 2012 at 7:35 PM, ioan...@btinternet.com wrote: > I am hoping someone can spot what is missing here. I am getting null result > from curl-ing a page with json post variables. > > I try this url in my Firefox browser - > > http://www.targetsite.com/search.php#somevar.someothervar > > (#somevar.someothervar are irrelevant, I think, as I get the curl variables > from Firebug below.) > > In Firebug, this shows: > > POST http://www.targetsite.com/ajax/search.xml > > In Firebug, below this link are tabs for: Headers, Post, Response, XML and > Cookies. Post tab shows like: > > JSON > VAR1 1 > VAR2 "2012-09-12" > VAR3 null > CACHED [] > OPTIONSnull > > To prove there is output, the Firebug Response tab shows: > > > various JSON encoded stuff > > The above is what I am trying to curl. > > My php code: > > $ch = curl_init(); > curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); > curl_setopt($ch, CURLOPT_POST, 1); > > //target page from Firebug above: > curl_setopt($ch, CURLOPT_URL, > "http://www.targetsite.com/ajax/search.xml";); > > //I was not sure how to represent CACHED [], so set it to null try "CACHED"=>array() > $data = array( > "VAR1" => 1, > "VAR2" => "2012-09-12", > "VAR3" => null, > "CACHED"=>null, > "OPTIONS"=>null, > ); > curl_setopt($ch, CURLOPT_POSTFIELDS, $data); > > //make the request > $result = curl_exec($ch); > > //this returns null > > Any ideas where to go with this? Maybe I need to include the Cookies? I use > the above php and curl functions normally so it's all installed on the > server. > > John It might be that the site is using sessions/cookies. Have a look at the header data with firebug. Not sure if that's the problem, to find out what's really going on, call echo curl_error($ch); after curl_exec to find out what went wrong exactly. If you still don't know how to proceed, paste the result of the curl_error call in your reply. - Matijn -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] PHP CURL JSON POST Firebug
I am hoping someone can spot what is missing here. I am getting null result from curl-ing a page with json post variables. I try this url in my Firefox browser - http://www.targetsite.com/search.php#somevar.someothervar (#somevar.someothervar are irrelevant, I think, as I get the curl variables from Firebug below.) In Firebug, this shows: POST http://www.targetsite.com/ajax/search.xml In Firebug, below this link are tabs for: Headers, Post, Response, XML and Cookies. Post tab shows like: JSON VAR1 1 VAR2 "2012-09-12" VAR3 null CACHED [] OPTIONSnull To prove there is output, the Firebug Response tab shows: various JSON encoded stuff The above is what I am trying to curl. My php code: $ch = curl_init(); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch, CURLOPT_POST, 1); //target page from Firebug above: curl_setopt($ch, CURLOPT_URL, "http://www.targetsite.com/ajax/search.xml";); //I was not sure how to represent CACHED [], so set it to null $data = array( "VAR1" => 1, "VAR2" => "2012-09-12", "VAR3" => null, "CACHED"=>null, "OPTIONS"=>null, ); curl_setopt($ch, CURLOPT_POSTFIELDS, $data); //make the request $result = curl_exec($ch); //this returns null Any ideas where to go with this? Maybe I need to include the Cookies? I use the above php and curl functions normally so it's all installed on the server. John -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
RES: [PHP] CURL -d
Hi try this :
$ch = curl_init();
curl_setopt ( $ch, CURLOPT_URL, "URL HERE" );
curl_setopt ( $ch, CURLOPT_FOLLOWLOCATION, 1 );
curl_setopt ( $ch, CURLOPT_HEADER, 1 );
curl_setopt ( $ch, CURLOPT_TIMEOUT, 10);
curl_setopt ( $ch, CURLOPT_SSL_VERIFYPEER, false );
curl_setopt ( $ch, CURLOPT_SSL_VERIFYHOST, false );
This to use SSL Certicade
curl_setopt ( $ch, CURLOPT_SSLCERT, Public Key );
curl_setopt ( $ch, CURLOPT_SSLCERTPASSWD, '');
curl_setopt ( $ch, CURLOPT_SSLCERTTYPE, 'PEM' ); --> Type o certificade
curl_setopt ( $ch, CURLOPT_SSLKEY, Private Key );
curl_setopt ( $ch, CURLOPT_SSLKEYPASSWD, Password of private key);
curl_setopt ( $ch, CURLOPT_RETURNTRANSFER, 1 );
curl_setopt ( $ch, CURLOPT_POST, true );
curl_setopt ( $ch, CURLOPT_PORT , 443 ); --> port
curl_setopt($ch, CURLOPT_POSTFIELDS, DATA HERE );
curl_setopt($ch, CURLOPT_HTTPHEADER,
array(
'User-Agent: Some name',
'Accept-Encoding: gzip, deflate',
'POST "some post data" HTTP/1.1',
'Host: some host',
'Content-type: application/soap+xml; charset="utf-8"', The content type
'Content-Length: ', length of data
// This for soap action
'SOAPAction: action here ')
);
$xResult = curl_exec($ch);
Alejandro M.S.
-Mensagem original-
De: QI.VOLMAR QI [mailto:qi.vol...@gmail.com]
Enviada em: segunda-feira, 26 de março de 2012 12:46
Para: PHP General
Assunto: [PHP] CURL -d
I have this lines:
curl -H "Content-Type: application/json" -d "hello world" \
"http://api.pusherapp.com/apps/17331/channels/test_channel/events?"\
The option -d is for data. But How I can set it on the PHP CURL extension?
I have found that if I set something like array('Hello Word', 'name'
=> 'my_name'), for the POST data may occurs. Can you give me a little help?
--
PHP General Mailing List (http://www.php.net/) To unsubscribe, visit:
http://www.php.net/unsub.php
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] CURL -d
I know guess that it is a POST field, but the detail is on simulate the -d
without a label. I've already looked at setopt man page, but nothing seems
like what I need.
2012/3/26 Ashley Sheridan
> **
> On Mon, 2012-03-26 at 09:45 -0600, QI.VOLMAR QI wrote:
>
> I have this lines:
> curl -H "Content-Type: application/json" -d "hello world" \
>
> "http://api.pusherapp.com/apps/17331/channels/test_channel/events?"\
> <http://api.pusherapp.com/apps/17331/channels/test_channel/events?%5C%3E>
>
> The option -d is for data. But How I can set it on the PHP CURL extension?
> I have found that if I set something like array('Hello Word', 'name'
> => 'my_name'), for the POST data may occurs. Can you give me a little
> help?
>
>
>
> Have a look at curl_setopt() which can set those flags as you need inside
> PHP:
>
> http://uk3.php.net/manual/en/function.curl-setopt.php
>
>
> --
> Thanks,
> Ash
> http://www.ashleysheridan.co.uk
>
>
>
Re: [PHP] CURL -d
On Mon, 2012-03-26 at 09:45 -0600, QI.VOLMAR QI wrote:
> I have this lines:
> curl -H "Content-Type: application/json" -d "hello world" \
>
> "http://api.pusherapp.com/apps/17331/channels/test_channel/events?"\
>
> The option -d is for data. But How I can set it on the PHP CURL extension?
> I have found that if I set something like array('Hello Word', 'name'
> => 'my_name'), for the POST data may occurs. Can you give me a little
> help?
>
Have a look at curl_setopt() which can set those flags as you need
inside PHP:
http://uk3.php.net/manual/en/function.curl-setopt.php
--
Thanks,
Ash
http://www.ashleysheridan.co.uk
[PHP] CURL -d
I have this lines:
curl -H "Content-Type: application/json" -d "hello world" \
"http://api.pusherapp.com/apps/17331/channels/test_channel/events?"\
The option -d is for data. But How I can set it on the PHP CURL extension?
I have found that if I set something like array('Hello Word', 'name'
=> 'my_name'), for the POST data may occurs. Can you give me a little
help?
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] cURL and SSL
Hi,
I want to change a SOAP call to curl due to ssl timeout bug of PHP Soap.
My PHP version is PHP 5.2.6-1+lenny13 with Suhosin-Patch 0.9.6.2. I'm
using the following SOAP call and it's handling SSL very good:
$client = new SoapClient('https://92.42.55.82:8442', array('trace' =>
True, 'local_cert' => 'BehnamCa/newkey.pem'));
But when i want to use cURL and overwrite the __doRequest function using
the following code:
$curl = curl_init($location);
curl_setopt($curl, CURLOPT_VERBOSE, TRUE);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($curl, CURLOPT_POST, TRUE);
curl_setopt($curl, CURLOPT_POSTFIELDS, $request);
curl_setopt($curl, CURLOPT_HEADER, FALSE);
curl_setopt($curl, CURLOPT_HTTPHEADER, array("Content-Type: text/xml"));
curl_setopt($curl, CURLOPT_TIMEOUT, $this->timeout);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, 1);
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, 1);
curl_setopt($curl, CURLOPT_CAPATH, '/home/sasan/CA2/BehnamCa/');
curl_setopt($curl, CURLOPT_SSLCERT, '/home/sasan/CA2/BehnamCa/newkey.pem');
curl_setopt($curl, CURLOPT_SSLCERTTYPE, 'PEM');
curl_setopt($curl, CURLOPT_SSLKEY, '/home/sasan/CA2/BehnamCa/private.key');
curl_setopt($curl, CURLOPT_CAINFO, '/home/sasan/CA2/BehnamCa/private.crt');
$response = curl_exec($curl);
I'm receiving the following:
* About to connect() to 92.X.X.X port 8443 (#0)
* Trying 92.X.X.X... * connected
* Connected to 92.X.X.X (92.X.X.X) port 8443 (#0)
* successfully set certificate verify locations:
* CAfile: /home/sasan/CA2/BehnamCa/private.crt
CApath: /home/sasan/CA2/BehnamCa/
* SSL certificate problem, verify that the CA cert is OK. Details:
error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate
verify failed
* Closing connection #0
string(146) "SSL certificate problem, verify that the CA cert is OK.
Details:
error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate
verify failed"
As you can see, i'm trying every possible way to tell curl about my
keys, but still no success. Am i doing something wrong or this is a bug?
--
Best Regards
Sasan Rose
signature.asc
Description: OpenPGP digital signature
Re: [PHP] curl equivalent in PHP
On Fri, Mar 2, 2012 at 6:09 PM, Nibin V M wrote:
> Hmm..I am a php newbie ( just started learning it )...
>
> what my need is to display website from my server always for a
> non-registered domain.This is the code I use now to display the website
>
> $opts = array(
> 'http'=>array(
> 'method'=>"GET",
> 'header'=>"Accept-language: en\r\n" .
> "Cookie: foo=bar\r\n"
> )
> );
>
> $context = stream_context_create($opts);
>
> $fp = fopen('http://www.blahblah.com', 'r', false, $context);
> fpassthru($fp);
> fclose($fp);
> ?>
>
> Of course blahblah.com isn't registered yet. This will load the index page
> of blahblah.com fine ( since I use hosts file to resolve blahblah.com on
> the server ). But if we click on any link, it will give "server not found"
> error since my "local machine" is trying to resolve blahblah.com this time.
> I need to avoid it and want to load my domain "always" from the server when
> I click any link on the webpage, etc .
>
> How can I modify the above code to make it work! If possible, I request
> somebody to tell me the exact code that I need to change in the above
> script - since most of the php part is still greek to me :)
>
> Thank you,
Have you actually read/tried Daniel Browns reply?
The following seems to be all you need...
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] curl equivalent in PHP
Hmm..I am a php newbie ( just started learning it )...
what my need is to display website from my server always for a
non-registered domain.This is the code I use now to display the website
array(
'method'=>"GET",
'header'=>"Accept-language: en\r\n" .
"Cookie: foo=bar\r\n"
)
);
$context = stream_context_create($opts);
$fp = fopen('http://www.blahblah.com', 'r', false, $context);
fpassthru($fp);
fclose($fp);
?>
Of course blahblah.com isn't registered yet. This will load the index page
of blahblah.com fine ( since I use hosts file to resolve blahblah.com on
the server ). But if we click on any link, it will give "server not found"
error since my "local machine" is trying to resolve blahblah.com this time.
I need to avoid it and want to load my domain "always" from the server when
I click any link on the webpage, etc .
How can I modify the above code to make it work! If possible, I request
somebody to tell me the exact code that I need to change in the above
script - since most of the php part is still greek to me :)
Thank you,
On Fri, Mar 2, 2012 at 10:09 PM, Micky Hulse wrote:
> Hello,
>
> On Fri, Mar 2, 2012 at 8:07 AM, Jim Lucas wrote:
> >> But on some of my servers, curl isn't enabled! Is there any equivalent
> >> code
> >> to achieve the same?
>
> I've used a combination of output buffering [1], readfile() [2] and
> caching (specific to the framework I was using).
>
> Simply, you could resort to:
>
> http://www.google.com'); ?>
>
> Ofc, the links would need to be absolute in order for the assets to load.
>
> Good luck!
>
> Cheers,
> Micky
>
>[1] php.net/ob-start
>[2] php.net/readfile
>
> --
> PHP General Mailing List (http://www.php.net/)
> To unsubscribe, visit: http://www.php.net/unsub.php
>
>
--
Regards
Nibin.
http://TechsWare.in
Re: [PHP] curl equivalent in PHP
On Fri, Mar 2, 2012 at 11:29, Nibin V M wrote: > Thanks guys :) > > Now another problem...I use a test domain ( say blahblah.com ) for testing > this, which isn't registered yet. So with the given code the index page is > loading fine. But when I try to click on any links, it will redirect to the > original domain which isn't exists! Modify it as needed, and if you want to surf the site using the script as a proxy, add the logic to handle that. -- Network Infrastructure Manager http://www.php.net/ -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] curl equivalent in PHP
Hello, On Fri, Mar 2, 2012 at 8:07 AM, Jim Lucas wrote: >> But on some of my servers, curl isn't enabled! Is there any equivalent >> code >> to achieve the same? I've used a combination of output buffering [1], readfile() [2] and caching (specific to the framework I was using). Simply, you could resort to: http://www.google.com'); ?> Ofc, the links would need to be absolute in order for the assets to load. Good luck! Cheers, Micky [1] php.net/ob-start [2] php.net/readfile -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] curl equivalent in PHP
Thanks guys :) Now another problem...I use a test domain ( say blahblah.com ) for testing this, which isn't registered yet. So with the given code the index page is loading fine. But when I try to click on any links, it will redirect to the original domain which isn't exists! ( I have actually pointed the test domain to my server via hosts file on the server ) So I wanted to browse the website via a PHP script, same as if I use a browser on the server. Hope you get me! :) Is there any way to achieve this via PHP? What I wanted to On Fri, Mar 2, 2012 at 9:37 PM, Jim Lucas wrote: > On 03/02/2012 06:26 AM, Nibin V M wrote: > >> Hello, >> >> I am trying to display the website content through a php code ( my own >> websites; doesn't cause copy right issues ). >> >> I use curl to display the page via the following simple code. >> >> > $curl = curl_init(); >> curl_setopt ($curl, CURLOPT_URL, "http://mytest.com";); >> curl_exec ($curl); >> curl_close ($curl); >> ?> >> >> But on some of my servers, curl isn't enabled! Is there any equivalent >> code >> to achieve the same? >> >> Thank you, >> >> > As long as you do not need to perform posts to the other website via the > PHP request, you could include, file_get_contents, fopen + fread, etc... > > All you need to make sure is that allow_url_fopen is enabled. > > -- > Jim Lucas > > http://www.cmsws.com/ > http://www.cmsws.com/examples/ > http://www.bendsource.com/ > -- Regards Nibin. http://TechsWare.in
Re: [PHP] curl equivalent in PHP
On 03/02/2012 06:26 AM, Nibin V M wrote: Hello, I am trying to display the website content through a php code ( my own websites; doesn't cause copy right issues ). I use curl to display the page via the following simple code. http://mytest.com";); curl_exec ($curl); curl_close ($curl); ?> But on some of my servers, curl isn't enabled! Is there any equivalent code to achieve the same? Thank you, As long as you do not need to perform posts to the other website via the PHP request, you could include, file_get_contents, fopen + fread, etc... All you need to make sure is that allow_url_fopen is enabled. -- Jim Lucas http://www.cmsws.com/ http://www.cmsws.com/examples/ http://www.bendsource.com/ -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] curl equivalent in PHP
http://www.php.net/manual/en/function.stream-context-create.php On Fri, Mar 2, 2012 at 4:44 PM, Nibin V M wrote: > Thanks Marc. But that need to add the DOM parser to the server. What I am > looking for something like "iframe" in html and that doesn't require any > additional PHP modules ( I do would like to avoid additions to the current > php; that is why I didn't compiled in curl ) > > On Fri, Mar 2, 2012 at 8:10 PM, Nibin V M wrote: > > > Thanks Marc. But that need to add the DOM parser to the server. What I am > > looking for something like "iframe" in html and that doesn't require any > > additional PHP modules ( I do would like to avoid additions to the > current > > php; that is why I didn't compiled in curl ) > > > > > > On Fri, Mar 2, 2012 at 7:58 PM, Marc Guay wrote: > > > >> http://simplehtmldom.sourceforge.net/ > >> > >> -- > >> PHP General Mailing List (http://www.php.net/) > >> To unsubscribe, visit: http://www.php.net/unsub.php > >> > >> > > > > > > -- > > Regards > > > > Nibin. > > > > http://TechsWare.in > > > > > > > -- > Regards > > Nibin. > > http://TechsWare.in >
Re: [PHP] curl equivalent in PHP
Thanks Marc. But that need to add the DOM parser to the server. What I am looking for something like "iframe" in html and that doesn't require any additional PHP modules ( I do would like to avoid additions to the current php; that is why I didn't compiled in curl ) On Fri, Mar 2, 2012 at 8:10 PM, Nibin V M wrote: > Thanks Marc. But that need to add the DOM parser to the server. What I am > looking for something like "iframe" in html and that doesn't require any > additional PHP modules ( I do would like to avoid additions to the current > php; that is why I didn't compiled in curl ) > > > On Fri, Mar 2, 2012 at 7:58 PM, Marc Guay wrote: > >> http://simplehtmldom.sourceforge.net/ >> >> -- >> PHP General Mailing List (http://www.php.net/) >> To unsubscribe, visit: http://www.php.net/unsub.php >> >> > > > -- > Regards > > Nibin. > > http://TechsWare.in > > -- Regards Nibin. http://TechsWare.in
Re: [PHP] curl equivalent in PHP
Thanks Marc. But that need to add the DOM parser to the server. What I am looking for something like "iframe" in html and that doesn't require any additional PHP modules ( I do would like to avoid additions to the current php; that is why I didn't compiled in curl ) On Fri, Mar 2, 2012 at 7:58 PM, Marc Guay wrote: > http://simplehtmldom.sourceforge.net/ > > -- > PHP General Mailing List (http://www.php.net/) > To unsubscribe, visit: http://www.php.net/unsub.php > > -- Regards Nibin. http://TechsWare.in
[PHP] curl equivalent in PHP
Hello, I am trying to display the website content through a php code ( my own websites; doesn't cause copy right issues ). I use curl to display the page via the following simple code. http://mytest.com";); curl_exec ($curl); curl_close ($curl); ?> But on some of my servers, curl isn't enabled! Is there any equivalent code to achieve the same? Thank you, -- Regards Nibin. http://TechsWare.in
Re: [PHP] Curl problems
On Thu, Jan 12, 2012 at 12:44 AM, Rick Dwyer wrote: > On Jan 11, 2012, at 6:29 PM, Matijn Woudt wrote: > >> On Thu, Jan 12, 2012 at 12:20 AM, Rick Dwyer >> wrote: >>> >>> Hello all. >>> >>> I use curl to make a call to another page on my site... but it operates >>> erroneously sometimes working... sometimes not. The page it calls >>> creates an email and I can see on the server the email in the queue when >>> it's working. If I echo out the URL the curl command is supposed to load >>> and load it manually, it works without fail. >>> >>> Any help on what I am doing wrong below is greatly appreciated. >>> >>> Thanks. >>> >>> >>> $curl_handle=curl_init(); >>> >>> curl_setopt($curl_handle,CURLOPT_URL,'https://mydomain.com/email_confirmation.htm?id_order='.$id_order.'&sess_id='.$sess_id); >>> curl_exec($curl_handle); >>> curl_close($curl_handle); >>> >>> --Rick >> >> >> It's maybe not a real answer to your question, but if all you want to >> do is call that page, why don't you just use >> >> file_get_contents(https://mydomain.com/email_confirmation.htm?id_order='.$id_order.'&sess_id='.$sess_id); >> (See [1]) >> It works out of the box, and I have found curl unstable too sometimes. >> >> Matijn > > > Thanks Matijn, > But I get "Notice: file_get_contents() [function.file-get-contents]: Unable > to find the wrapper "https" - did you forget to enable it when you > configured PHP?"... I'm using a hosting provider and I don't believe they > will enable this for security reasons. > > --Rick It seems like they have not compiled PHP with SSL support, or they're using a pretty old version. Anyway, you're probably stuck with cURL then, check the return of curl_exec, and if false, call curl_error to get an error message. Matijn -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Curl problems
On Jan 11, 2012, at 6:29 PM, Matijn Woudt wrote: On Thu, Jan 12, 2012 at 12:20 AM, Rick Dwyer wrote: Hello all. I use curl to make a call to another page on my site... but it operates erroneously sometimes working... sometimes not. The page it calls creates an email and I can see on the server the email in the queue when it's working. If I echo out the URL the curl command is supposed to load and load it manually, it works without fail. Any help on what I am doing wrong below is greatly appreciated. Thanks. $curl_handle=curl_init(); curl_setopt($curl_handle,CURLOPT_URL,'https://mydomain.com/email_confirmation.htm?id_order='.$id_order.'&sess_id='.$sess_id) ; curl_exec($curl_handle); curl_close($curl_handle); --Rick It's maybe not a real answer to your question, but if all you want to do is call that page, why don't you just use file_get_contents(https://mydomain.com/email_confirmation.htm?id_order='.$id_order.'&sess_id='.$sess_id ); (See [1]) It works out of the box, and I have found curl unstable too sometimes. Matijn Thanks Matijn, But I get "Notice: file_get_contents() [function.file-get-contents]: Unable to find the wrapper "https" - did you forget to enable it when you configured PHP?"... I'm using a hosting provider and I don't believe they will enable this for security reasons. --Rick -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Curl problems
On Thu, Jan 12, 2012 at 12:20 AM, Rick Dwyer wrote: > Hello all. > > I use curl to make a call to another page on my site... but it operates > erroneously sometimes working... sometimes not. The page it calls > creates an email and I can see on the server the email in the queue when > it's working. If I echo out the URL the curl command is supposed to load > and load it manually, it works without fail. > > Any help on what I am doing wrong below is greatly appreciated. > > Thanks. > > > $curl_handle=curl_init(); > curl_setopt($curl_handle,CURLOPT_URL,'https://mydomain.com/email_confirmation.htm?id_order='.$id_order.'&sess_id='.$sess_id); > curl_exec($curl_handle); > curl_close($curl_handle); > > --Rick It's maybe not a real answer to your question, but if all you want to do is call that page, why don't you just use file_get_contents(https://mydomain.com/email_confirmation.htm?id_order='.$id_order.'&sess_id='.$sess_id); (See [1]) It works out of the box, and I have found curl unstable too sometimes. Matijn [1] www.php.net/file_get_contents -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] Curl problems
Hello all. I use curl to make a call to another page on my site... but it operates erroneously sometimes working... sometimes not. The page it calls creates an email and I can see on the server the email in the queue when it's working. If I echo out the URL the curl command is supposed to load and load it manually, it works without fail. Any help on what I am doing wrong below is greatly appreciated. Thanks. $curl_handle=curl_init(); curl_setopt($curl_handle,CURLOPT_URL,'https://mydomain.com/email_confirmation.htm?id_order='.$id_order.'&sess_id='.$sess_id) ; curl_exec($curl_handle); curl_close($curl_handle); --Rick -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] curl ,response headers and gzip encoding
hi i am trying to build a proxy server for opera's turbo mod
$value) {
$data.= "\n$name: $value";
}
$data=str_replace('Host: w.com','Host: '.$domainadi,$data);
$data.="\r\n\r\n";
if($_SERVER['REQUEST_METHOD']=='POST') {
$data.=@file_get_contents('php://input'); }
#echo $domain;exit;
#echo $data;exit;
$ch = curl_init($domain);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
if($_SERVER['REQUEST_METHOD']=='POST') { curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, @file_get_contents('php://input')); }
$header = array();
foreach(getallheaders() as $key => $value)
{
if(strtolower($key)!='host')
{
$header[] = $key . ':' . $value;
}
}
curl_setopt($ch , CURLOPT_HTTPHEADER , $header);
$donen=@curl_exec($ch);
#header('Content-Encoding: gzip');
if(curl_getinfo($ch,CURLINFO_HTTP_CODE)=='0') die('Hata Kod:36');
@list($reshead, $body) = @explode("\r\n\r\n", $donen, 2) or die('Hata Kod
:37');
$info = curl_getinfo($ch);
$reshead = substr($donen, 0, $info['header_size']);
$body = substr($donen, $info['header_size']);
$headerlar=explode("\n",$reshead);
for($i=0;$i
it returns the gzip content as damaged may be for split the headers and
body ,
,
when i take curl_setopt($ch, CURLOPT_HEADER, 0); to curl_setopt($ch,
CURLOPT_HEADER, 1);
header('Content-Encoding: gzip'); uncomment this line
hide these lines
/*
if(curl_getinfo($ch,CURLINFO_HTTP_CODE)=='0') die('Hata Kod:36');
@list($reshead, $body) = @explode("\r\n\r\n", $donen, 2) or die('Hata Kod
:37');
$info = curl_getinfo($ch);
$reshead = substr($donen, 0, $info['header_size']);
$body = substr($donen, $info['header_size']);
$headerlar=explode("\n",$reshead);
for($i=0;$i
Re: [PHP] Curl cost
On Wed, Sep 28, 2011 at 4:06 PM, tamouse mailing lists wrote: > On Wed, Sep 28, 2011 at 3:28 AM, muad shibani wrote: >> what are the costs of using PHP Curl to show another websites on my site as >> stumbleon do .. >> traffic, memory or what? > > If you use curl to suck the web page into a variable, could be > tremendous. Better to curl it into a temporary file if you're going > that way. > Just a followup -- if you want to show other websites on your own page, a friendly way to do this is using an iframe: http://www.example.com";; ?> Your browser does not support iframes That way, the site gets the hit, and you aren't copying anything. If you do go this way, make sure to provide a break out so the user can open the target site in a new window/tab. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Fwd: [PHP] Curl cost
Forgot to include list... -- Forwarded message -- From: tamouse mailing lists Date: Wed, Sep 28, 2011 at 4:06 PM Subject: Re: [PHP] Curl cost To: muad shibani On Wed, Sep 28, 2011 at 3:28 AM, muad shibani wrote: > what are the costs of using PHP Curl to show another websites on my site as > stumbleon do .. > traffic, memory or what? If you use curl to suck the web page into a variable, could be tremendous. Better to curl it into a temporary file if you're going that way. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Curl cost
On Wed, Sep 28, 2011 at 14:13, Jim Lucas wrote: > > I saw that, but @ss-umed it was a typo. My bad. I'd presumed the same at first. You're in good company still. -- Network Infrastructure Manager http://www.php.net/ -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Curl cost
On 9/28/2011 9:05 AM, Daniel Brown wrote: > On Wed, Sep 28, 2011 at 11:54, Jim Lucas wrote: >> >> That isn't how stumbleupon does it. You might want to take a closer look at >> the >> HTML to see how they do what they do. > > He said stumbleON, actually. Looks like they simply aggregate > some of your personal social networking data into a single-sign-on > dashboard presentation. Certainly no service I'd use, but I suppose > for some it has its merits. > I saw that, but @ss-umed it was a typo. My bad. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Curl cost
On Wed, Sep 28, 2011 at 11:54, Jim Lucas wrote: > > That isn't how stumbleupon does it. You might want to take a closer look at > the > HTML to see how they do what they do. He said stumbleON, actually. Looks like they simply aggregate some of your personal social networking data into a single-sign-on dashboard presentation. Certainly no service I'd use, but I suppose for some it has its merits. -- Network Infrastructure Manager http://www.php.net/ -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Curl cost
On 9/28/2011 1:28 AM, muad shibani wrote: > what are the costs of using PHP Curl to show another websites on my site as > stumbleon do .. > traffic, memory or what? > That isn't how stumbleupon does it. You might want to take a closer look at the HTML to see how they do what they do. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] Curl cost
what are the costs of using PHP Curl to show another websites on my site as stumbleon do .. traffic, memory or what?
Re: [PHP] curl & rtmp
On Sun, Jan 9, 2011 at 2:58 PM, Tontonq Tontonq wrote: > does cUrl supports rtmp protocol? if so is there any example? These are obvious by searching for the terms, which seem to be quite specific to have not found an answer in the search engines. do we need > enable different library? so if not can we save rtmp by curl? if not is > there any other rtmp downloader that u know ? You seem to know enough to have answered this by yourself, almost in your own questions. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] curl & rtmp
does cUrl supports rtmp protocol? if so is there any example? do we need enable different library? so if not can we save rtmp by curl? if not is there any other rtmp downloader that u know ?
RE: [PHP] curl and variable parameters in hyperlink
Thanks to all for your interest so far but as I said, new to PHP. Changed POST to GET and all now working fine! Tommy, the third party site is fully aware of what I am doing. We pay them to use the data, as do lots of other companies in our industry. Supplying data is what they do. We could not use the site at all without being given a valid user name. Far from 'preventing them being successful' the more users they have the more successful they become. We do not 'pass the work as our own' they are credited on our site as suppliers of the content, so there really is no issue there. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP] curl and variable parameters in hyperlink
> -Original Message- > From: Bob Keightley [mailto:bob.keight...@virgin.net] > Sent: Wednesday, November 24, 2010 11:25 AM > To: php-general@lists.php.net > Subject: Re: [PHP] curl and variable parameters in hyperlink > > Guess I've not explained this very well. > > The external page I am fetching using another curl script has asp query > hyperlinks in it. I do not know the variable names in each query or the > values they have. > > All I want to do is replace the external url referenced in those links with a > url which is on my site (so that I can change style sheet, image locations > etc.) but the parameters to the original asp query link have to be captured > and passed to my php script for it to return data. I know there must be a > way of doing this as there's a site which is referencing the same third party > site and doing the same thing, and no, they won't tell me how! > Bob, There's several issues with this. 1) They're providing the info which you're trying to pass it as your own and most likely that they won't get any positive results, as some company do analysis on visits to improve the marketing strategy, customer service, customer relations, etc. You're just skewing their data analysis and prevent them from becoming successful. Which may force them to go out of business then you'll lose your source of information. 2) Waste of bandwidth. 3rd party > you > user instead of just you > user. 3) I'm don't like that fact that some people take others work and try to pass it as their own. I think many folks on this list feels the same way. So why not just quote them and give them the credit they deserve. That's probably the reason why you're not able to find much info on how to do this :) Regards, Tommy -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] curl and variable parameters in hyperlink
Guess I've not explained this very well. The external page I am fetching using another curl script has asp query hyperlinks in it. I do not know the variable names in each query or the values they have. All I want to do is replace the external url referenced in those links with a url which is on my site (so that I can change style sheet, image locations etc.) but the parameters to the original asp query link have to be captured and passed to my php script for it to return data. I know there must be a way of doing this as there's a site which is referencing the same third party site and doing the same thing, and no, they won't tell me how! -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] curl and variable parameters in hyperlink
On Wednesday 24 November 2010,
"Bob Keightley" wrote:
> I already have a curl script that gets the web page, but it doesn't pass
> the parameters
Hello Bob,
>
> Being new to PHP I haven't the first idea how to modify it so that it
> does.
>
> Script is as follows:
>
> $url = "http://www.xx.com/query.asp?param1=val1¶m2=val2";;
>
> foreach ($_POST as $key=>$post) {
> $post=str_replace(" ", "+", $post);
> $url.=$key."=".$post."&";
> }
Instead of concatenating strings, I suggest to use the http_build_query()
function:
'val1',
'param2' => 'val2',
'param3' => 'val3');
$query = http_build_query($data);
echo "Query: {$query}\n";
?>
So, to create a query string based on your $_POST request parameters,
you just need to use the function as follows:
This will create the proper query string to be used on your URL.
>
> $ch = curl_init($url);
> curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
> curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);
> $data = curl_exec($ch);
> curl_close($ch);
>
> $data=str_replace('.asp', '.php', $data);
> echo $data;
>
> This returns the web page, but ignores val1 and val2 which are necessary
> to execute the query.
Best regards,
--
Daniel Molina Wegener
System Programmer & Web Developer
Phone: +56 (2) 979-0277 | Blog: http://coder.cl/
signature.asc
Description: This is a digitally signed message part.
Re: [PHP] curl and variable parameters in hyperlink
>
> foreach ($_POST as $key=>$post) {
>$post=str_replace(" ", "+", $post);
>$url.=$key."=".$post."&";
>}
>
Hi Bob,
One thing I see is that you're appending values on to the end of a url that
already has 2 values, so you should place the "&" at the beginning of your
line instead of at the end. Value 2 is being merged with whatever value is
coming after it in your example. So, try something like:
$url .= "&" . $key. "=" .$post;
Also, I'd suggest url encoding the post value rather than merely hunting
down spaces and swapping them out with "+".
Hope this helps,
Adam
--
Nephtali: PHP web framework that functions beautifully
http://nephtaliproject.com
RE: [PHP] curl and variable parameters in hyperlink
I already have a curl script that gets the web page, but it doesn't pass the
parameters
Being new to PHP I haven't the first idea how to modify it so that it does.
Script is as follows:
$url = "http://www.xx.com/query.asp?param1=val1¶m2=val2";;
foreach ($_POST as $key=>$post) {
$post=str_replace(" ", "+", $post);
$url.=$key."=".$post."&";
}
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);
$data = curl_exec($ch);
curl_close($ch);
$data=str_replace('.asp', '.php', $data);
echo $data;
This returns the web page, but ignores val1 and val2 which are necessary to
execute the query.
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP] curl and variable parameters in hyperlink
Depending on how the website is setup you can curl or soap parameters. http://www.php.net/manual/en/book.curl.php http://www.php.net/manual/en/book.soap.php Richard L. Buskirk -Original Message- From: Bob Keightley [mailto:bob.keight...@virgin.net] Sent: Tuesday, November 23, 2010 9:12 PM To: php-general@lists.php.net Subject: [PHP] curl and variable parameters in hyperlink First use of Curl so probably a very simple problem - I have a script which loads a third party web page into my site. That page contains links which have variable parameters i.e. www.blahblah.asp?param1=xxx¶m2=yyy. Any clues as to what I need to do to pass these parameters in a curl script appreciated. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] curl and variable parameters in hyperlink
First use of Curl so probably a very simple problem - I have a script which loads a third party web page into my site. That page contains links which have variable parameters i.e. www.blahblah.asp?param1=xxx¶m2=yyy. Any clues as to what I need to do to pass these parameters in a curl script appreciated. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] Re: Snoopy port using PHP cURL library
> Does anyone know if the Snoopy class has been ported to use the > built-in PHP cURL library and released publicly somewhere? I converted it myself. If anyone's interested the class is attached. Marc -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] Snoopy port using PHP cURL library
Does anyone know if the Snoopy class has been ported to use the built-in PHP cURL library and released publicly somewhere? Marc -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] curl
# yum install php4-curl or # yum install php-curl Shiplu Mokadd.im My talks, http://talk.cmyweb.net Follow me, http://twitter.com/shiplu SUST Programmers, http://groups.google.com/group/p2psust Innovation distinguishes bet ... ... (ask Steve Jobs the rest) -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] curl
I have an old unix fedora system with php 4.3.11, with the following recompiled options $ php -version PHP 4.3.11 (cli) (built: Jun 6 2006 16:20:00) Copyright (c) 1997-2004 The PHP Group Zend Engine v1.3.0, Copyright (c) 1998-2004 Zend Technologies $ php -info | grep configure Configure Command => './configure' '--enable-discard-path' '--enable-track-vars' '--enable-force-cgi-redirect' '--with-gettext' '--with-mysql' '--enable-so' '--with-apxs2' '--with-gd' '--with-zlib' '--with-jpeg-dir=/usr/lib' '--enable-libgcc' am planning to add the curl option to it, what would be the best approach for that. stop http, configure with--curl, make, make install feedback appreciated. Thanks -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] curl help or other suggestion
As it turns out curl_multi does not block. I was able to use that plus connection tracking to solve my problem. Thanks Ash and Andrew. Mike > On Thu, Jun 17, 2010 at 9:05 AM, Michael Alaimo > wrote: >> I am trying to use register_shutdown_function. Â Previous to the script >> shutting down I use curl to grab a website. >> >> Basically I want to know if the user has hit the stop button or left the >> page prematurely. >> >> The only problem is that curl runs and finishes its call before the >> shutdown function is called. Â Because of this I Â have no way to know >> if >> the user canceled the request somehow. Â Any suggestions? >> >> Mike > > I don't know that it will affect cURL, but if you are just looking to > log completed versus abandoned requests, this might help: > > http://www.php.net/manual/en/function.ignore-user-abort.php > > Andrew > > -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] curl help or other suggestion
On Thu, Jun 17, 2010 at 9:05 AM, Michael Alaimo wrote: > I am trying to use register_shutdown_function. Previous to the script > shutting down I use curl to grab a website. > > Basically I want to know if the user has hit the stop button or left the > page prematurely. > > The only problem is that curl runs and finishes its call before the > shutdown function is called. Because of this I have no way to know if > the user canceled the request somehow. Any suggestions? > > Mike I don't know that it will affect cURL, but if you are just looking to log completed versus abandoned requests, this might help: http://www.php.net/manual/en/function.ignore-user-abort.php Andrew -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] curl help or other suggestion
On Thu, 2010-06-17 at 09:37 -0400, Michael Alaimo wrote: > > On Thu, 2010-06-17 at 09:05 -0400, Michael Alaimo wrote: > > > >> I am trying to use register_shutdown_function. Previous to the script > >> shutting down I use curl to grab a website. > >> > >> Basically I want to know if the user has hit the stop button or left the > >> page prematurely. > >> > >> The only problem is that curl runs and finishes its call before the > >> shutdown function is called. Because of this I have no way to know if > >> the user canceled the request somehow. Any suggestions? > >> > >> Mike > >> > >> > >> > > > > > > You can't. When a visitor closes their browser, or stops the page from > > completing its loading, no message is sent to the server, so your PHP > > script won't know about it. > > > > The only way I know of currently to check for this is use some form of > > regular check-in, with Ajax or something, that calls home at regular > > intervals to inform your script that it is indeed still connected. If it > > fails to call home in time it either means there's lag on the connection > > or they broke it. > > > > Thanks, > > Ash > > http://www.ashleysheridan.co.uk > > > > > I wish that curl would just stop processing. I think you can use > connection tracking to know what the user has done. I just cannot use it > with my current situation as the code is. > > Mike > If this situation is occurring often, maybe a re-think of the structure of your app. What is cURL fetching? Is it something that you can cache at your end, like an RSS feed, etc? If so, maybe use the local cache and only refresh if the local copy is too old. Thanks, Ash http://www.ashleysheridan.co.uk
Re: [PHP] curl help or other suggestion
> On Thu, 2010-06-17 at 09:05 -0400, Michael Alaimo wrote: > >> I am trying to use register_shutdown_function. Previous to the script >> shutting down I use curl to grab a website. >> >> Basically I want to know if the user has hit the stop button or left the >> page prematurely. >> >> The only problem is that curl runs and finishes its call before the >> shutdown function is called. Because of this I have no way to know if >> the user canceled the request somehow. Any suggestions? >> >> Mike >> >> >> > > > You can't. When a visitor closes their browser, or stops the page from > completing its loading, no message is sent to the server, so your PHP > script won't know about it. > > The only way I know of currently to check for this is use some form of > regular check-in, with Ajax or something, that calls home at regular > intervals to inform your script that it is indeed still connected. If it > fails to call home in time it either means there's lag on the connection > or they broke it. > > Thanks, > Ash > http://www.ashleysheridan.co.uk > I wish that curl would just stop processing. I think you can use connection tracking to know what the user has done. I just cannot use it with my current situation as the code is. Mike -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] curl help or other suggestion
On Thu, 2010-06-17 at 09:05 -0400, Michael Alaimo wrote: > I am trying to use register_shutdown_function. Previous to the script > shutting down I use curl to grab a website. > > Basically I want to know if the user has hit the stop button or left the > page prematurely. > > The only problem is that curl runs and finishes its call before the > shutdown function is called. Because of this I have no way to know if > the user canceled the request somehow. Any suggestions? > > Mike > > > You can't. When a visitor closes their browser, or stops the page from completing its loading, no message is sent to the server, so your PHP script won't know about it. The only way I know of currently to check for this is use some form of regular check-in, with Ajax or something, that calls home at regular intervals to inform your script that it is indeed still connected. If it fails to call home in time it either means there's lag on the connection or they broke it. Thanks, Ash http://www.ashleysheridan.co.uk
[PHP] curl help or other suggestion
I am trying to use register_shutdown_function. Previous to the script shutting down I use curl to grab a website. Basically I want to know if the user has hit the stop button or left the page prematurely. The only problem is that curl runs and finishes its call before the shutdown function is called. Because of this I have no way to know if the user canceled the request somehow. Any suggestions? Mike -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] CURL cannot connect to URL - IP address - after successful connection
On 4/29/10, ioan...@btinternet.com wrote: > On 2010/04/29 19:46, Gary . wrote: >> Failed to connect to host is a pretty strange error if they're doing >> anything regarding cookies and so on, IMO - I think I'd expect at >> least a connection to be established before they decide they don't >> like you. Have you used curl's --trace& --trace-ascii options? > > Is that debug_backtrace() in php Not sure :-P > as I am not using the command line > (can't work out how to get the window up having downloaded curl, I am > not up to 'building libraries' that seems to be needed). Windows? http://curl.haxx.se/download.html *n*x variants should allow installing via their packge management systems. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] CURL cannot connect to URL - IP address - after successful connection
On 2010/04/29 19:46, Gary . wrote: On 4/25/10, ioan...@btinternet.com wrote: I can return a target page - once, but then on refresh within a few hours the script curl_error is that it cannot connect to the host and return is empty. Failed to connect to host is a pretty strange error if they're doing anything regarding cookies and so on, IMO - I think I'd expect at least a connection to be established before they decide they don't like you. Have you used curl's --trace& --trace-ascii options? Is that debug_backtrace() in php, as I am not using the command line (can't work out how to get the window up having downloaded curl, I am not up to 'building libraries' that seems to be needed). debug_backtrace() does not give any useful information other than saying the target link fails to connect (this is after it connects once, and then on refresh and for several hours does not connect). I guess there is some program that notes the calling IP address and if it is in a range it does not like, adds it to a list and refuses subsequent connections to the same address for a while. Cookies are not required when using the browser directly. John -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] CURL cannot connect to URL - IP address - after successful connection
On 4/25/10, ioan...@btinternet.com wrote: > I can return a target page - once, but then on refresh within a few > hours the script curl_error is that it cannot connect to the host and > return is empty. Failed to connect to host is a pretty strange error if they're doing anything regarding cookies and so on, IMO - I think I'd expect at least a connection to be established before they decide they don't like you. Have you used curl's --trace & --trace-ascii options? -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP] CURL cannot connect to URL - IP address - after successful connection
> -Original Message- > From: ioan...@btinternet.com [mailto:ioan...@btinternet.com] > Sent: Wednesday, April 28, 2010 7:03 AM > To: 'PHP' > Subject: Re: [PHP] CURL cannot connect to URL - IP address - after successful > connection > > I think the answer is: ISPs have a different range of addresses from host > providers, so it is possible to block requests from host servers, so from > scripts. > > John > That's possible but very unlikely in your case. Since you were to able to get the necessary information on the 1st request, but failed on subsequent means it has an anti-bot mechanism in place. Bypassing anti-bot requires in-depth reverse-engineering of the targeted site. Regards, Tommy -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] CURL cannot connect to URL - IP address - after successful connection
I think the answer is: ISPs have a different range of addresses from host providers, so it is possible to block requests from host servers, so from scripts. John -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP] CURL cannot connect to URL - IP address - after successful connection
> -Original Message- > From: ioan...@btinternet.com [mailto:ioan...@btinternet.com] > Sent: Monday, April 26, 2010 7:10 AM > To: Tommy Pham > Subject: Re: [PHP] CURL cannot connect to URL - IP address - after successful > connection > > > > On 2010/04/27 1:13, Tommy Pham wrote: > >> -Original Message- > > > I assume that you did full testing with the browser as I suggested? > > If everything works, one other thing to keep in mind is that the > > target also may implement reverse DNS lookup in their anti-bot. One > > good way to test that is to remote in via SSH (if on Linux/Unix) to test with > wget. > > Otherwise, I'm pretty sure that target site have some anti-bot > > mechanisms in place. Microseconds of analyzing valid 'user' requests > > is better than processing 2-3 seconds and sending the response which > > will consume bandwidth. What you could also try is setting different > > user-agents for every request and use cookies in cURL should the > > target site have an anti-bot mechanism. > > > > Regards, > > Tommy > > > > > Yes, I think I have tested with/without cookies on the browser, trying > different user agents (code emailed previously using array and rand) and > cookies are used in script/not used. And it works on subsequent requests? > > I cannot work out how to use Putty/ssh/public private keys etc..wget... > Learning how to use that is easier than learning to code PHP, IMO. > I read about some problems with curl setting the port and a required patch > on the server. > > John If cURL requires some kind of patch as you say, then it wouldn't have work in the first place. Perhaps it's better to post your (obfuscated personal data) code. Or try on your local PC on your local web server to eliminate possibility of proxies, anti-bots, etc... problems to test that your code works as intended and not cURL problem as you say. I didn't have problems using cURL before. But then my targeted sites were very big companies and didn't care about bots much. Regards, Tommy PS: Always reply to the list so others in the future can benefit unless it's something personal. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP] CURL cannot connect to URL - IP address - after successful connection
> -Original Message- > From: ioan...@btinternet.com [mailto:ioan...@btinternet.com] > Sent: Sunday, April 25, 2010 10:44 PM > To: a...@ashleysheridan.co.uk; tommy...@gmail.com >> Tommy Pham > Subject: Re: [PHP] CURL cannot connect to URL - IP address - after successful > connection > > The answer I got from support desk on my shared server: 'You are trying to > curl to a datapipe server, if it is rejecting the server name and port, you will > need to take that up with them.' > > John I assume that you did full testing with the browser as I suggested? If everything works, one other thing to keep in mind is that the target also may implement reverse DNS lookup in their anti-bot. One good way to test that is to remote in via SSH (if on Linux/Unix) to test with wget. Otherwise, I'm pretty sure that target site have some anti-bot mechanisms in place. Microseconds of analyzing valid 'user' requests is better than processing 2-3 seconds and sending the response which will consume bandwidth. What you could also try is setting different user-agents for every request and use cookies in cURL should the target site have an anti-bot mechanism. Regards, Tommy -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] CURL cannot connect to URL - IP address - after successful connection
On 2010/04/26 20:01, Ashley Sheridan wrote: How frequently do you request the page? Maybe playing about with that would resolve it? Is it possible to randomise the request frequency a bit? Thanks, Ash http://www.ashleysheridan.co.uk Just manually for testing, and it would be used for human requests. Say occasionally 5, 10, 30 minutes intervals etc. There must be other parameters that are being passed so that the site can determine that the request is coming from the same user and through a script request, because it works normally from the browser so just refusing a second call from the same IP address (which could be a browser with static or unchanged IP address) is not what is happening. It must be determining that it is through a server from another site via curl or similar. John -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] CURL cannot connect to URL - IP address - after successful connection
On Mon, 2010-04-26 at 12:05 +0900, ioan...@btinternet.com wrote:
> >>
> >> Just to eliminate all possibilities, are you to open the same URL/URI in
> > the
> >> web pages repeatedly? Also, what happens when you fake the user agent in
> >> the web browser? The target site may have some anti bot mechanism in
> >> place to reduce stress/load on the server(s).
> >>
> >> Regards,
> >> Tommy
> >
> > One more thing, check it with cookies enabled/disabled in the web browser
> > too.
> >
> >
>
> Having deleted cookies on the browser and disabled them, it still does
> not like various user agents:
>
> $useragent = array('Mozilla','Opera','Microsoft Internet
> Explorer','ia_archiver');
> $os = array('Windows','Windows XP','Linux','Windows NT','Windows
> 2000','OSX');
> //random user agent code
> $agent = $useragent[rand(0,3)].'/'.rand(1,8).'.'.rand(0,9).'
> ('.$os[rand(0,5)].' '.rand(1,7).'.'.rand(0,9).'; en-US;)';
> //would give something like Mozilla/3.5 (Windows 5.4; en-US;)
>
> -- OR --
>
> //$useragent='Google Image - Googlebot-Image/1.0 (
> http://www.googlebot.com/bot.html)';
> //$useragent="MSN Live - msnbot-Products/1.0
> (+http://search.msn.com/msnbot.htm)";
>
> -- OR --
> //$agent = "DocZilla/1.0 (Windows; U; WinNT4.0; en-US; rv:1.0.0)
> Gecko/20020804";
>
> I am just calling the page manually, once at a time. It is probable
> that there is some anti-bot measures. Page would probably not want to
> be indexed as it is providing ever changing content. How to use this
> for normal level of use for real user just in a different site?
>
> John
>
How frequently do you request the page? Maybe playing about with that
would resolve it? Is it possible to randomise the request frequency a
bit?
Thanks,
Ash
http://www.ashleysheridan.co.uk
Re: [PHP] CURL cannot connect to URL - IP address - after successful connection
Just to eliminate all possibilities, are you to open the same URL/URI in
the
web pages repeatedly? Also, what happens when you fake the user agent in
the web browser? The target site may have some anti bot mechanism in
place to reduce stress/load on the server(s).
Regards,
Tommy
One more thing, check it with cookies enabled/disabled in the web browser
too.
Having deleted cookies on the browser and disabled them, it still does
not like various user agents:
$useragent = array('Mozilla','Opera','Microsoft Internet
Explorer','ia_archiver');
$os = array('Windows','Windows XP','Linux','Windows NT','Windows
2000','OSX');
//random user agent code
$agent = $useragent[rand(0,3)].'/'.rand(1,8).'.'.rand(0,9).'
('.$os[rand(0,5)].' '.rand(1,7).'.'.rand(0,9).'; en-US;)';
//would give something like Mozilla/3.5 (Windows 5.4; en-US;)
-- OR --
//$useragent='Google Image - Googlebot-Image/1.0 (
http://www.googlebot.com/bot.html)';
//$useragent="MSN Live - msnbot-Products/1.0
(+http://search.msn.com/msnbot.htm)";
-- OR --
//$agent = "DocZilla/1.0 (Windows; U; WinNT4.0; en-US; rv:1.0.0)
Gecko/20020804";
I am just calling the page manually, once at a time. It is probable
that there is some anti-bot measures. Page would probably not want to
be indexed as it is providing ever changing content. How to use this
for normal level of use for real user just in a different site?
John
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP] CURL cannot connect to URL - IP address - after successful connection
> -Original Message- > From: Tommy Pham [mailto:tommy...@gmail.com] > Sent: Monday, April 26, 2010 1:59 AM > To: 'php-general@lists.php.net' > Subject: RE: [PHP] CURL cannot connect to URL - IP address - after successful > connection > > > -Original Message- > > From: ioan...@btinternet.com [mailto:ioan...@btinternet.com] > > Sent: Sunday, April 25, 2010 6:18 AM > > To: php-general@lists.php.net > > Subject: [PHP] CURL cannot connect to URL - IP address - after > > successful connection > > > > I can return a target page - once, but then on refresh within a few > > hours the script curl_error is that it cannot connect to the host and return is > empty. > > The target URL is an ip address, not a named url, so maybe it has > > something to do with DNS. I am on a shared server. Any ideas on why this > happens? > > > > John > > > > Just to eliminate all possibilities, are you to open the same URL/URI in the > web pages repeatedly? Also, what happens when you fake the user agent in > the web browser? The target site may have some anti bot mechanism in > place to reduce stress/load on the server(s). > > Regards, > Tommy One more thing, check it with cookies enabled/disabled in the web browser too. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP] CURL cannot connect to URL - IP address - after successful connection
> -Original Message- > From: ioan...@btinternet.com [mailto:ioan...@btinternet.com] > Sent: Sunday, April 25, 2010 6:18 AM > To: php-general@lists.php.net > Subject: [PHP] CURL cannot connect to URL - IP address - after successful > connection > > I can return a target page - once, but then on refresh within a few hours the > script curl_error is that it cannot connect to the host and return is empty. > The target URL is an ip address, not a named url, so maybe it has something > to do with DNS. I am on a shared server. Any ideas on why this happens? > > John > Just to eliminate all possibilities, are you to open the same URL/URI in the web pages repeatedly? Also, what happens when you fake the user agent in the web browser? The target site may have some anti bot mechanism in place to reduce stress/load on the server(s). Regards, Tommy -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] CURL cannot connect to URL - IP address - after successful connection
This is all I see in the error log: SUEXEC error_log: [2010-04-25 16:45:42]: uid: (1116/myname) gid: (1118/myname) cmd: fcgiwrapper John -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] CURL cannot connect to URL - IP address - after successful connection
On Sun, 2010-04-25 at 22:17 +0900, ioan...@btinternet.com wrote: > I can return a target page - once, but then on refresh within a few > hours the script curl_error is that it cannot connect to the host and > return is empty. The target URL is an ip address, not a named url, so > maybe it has something to do with DNS. I am on a shared server. Any > ideas on why this happens? > > John > No, DNS is a Domain Named Server used to turn a domain name into an IP address. As you say you're using an IP address directly, it won't go near DNS. Are there any messages in the logs that would give more specific information? Thanks, Ash http://www.ashleysheridan.co.uk
[PHP] CURL cannot connect to URL - IP address - after successful connection
I can return a target page - once, but then on refresh within a few hours the script curl_error is that it cannot connect to the host and return is empty. The target URL is an ip address, not a named url, so maybe it has something to do with DNS. I am on a shared server. Any ideas on why this happens? John -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] cURL on freebsd
On Mar 30, 2010, at 19:50, Tommy Pham wrote: On Tue, Mar 30, 2010 at 7:36 PM, Shane Hill wrote: can anyone tell me why the cURL code below produces a POST request as expected on linux, but the same code on freebsd is forced to be a GET request? the linux box is running php 5.2.11 the freebsd box is running 5.3.2 is there a bug in 5.3.2? a search does not turn anything up. Just curious, what version of FreeBSD? Did you compile PHP & extensions or used the packages? Freebsd 7.2 stable I compiled everything enohPi ym morf tnes -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] cURL on freebsd
On Tue, Mar 30, 2010 at 7:36 PM, Shane Hill wrote: > can anyone tell me why the cURL code below produces a POST request as > expected on linux, but the same code on freebsd is forced to be a GET > request? > > the linux box is running php 5.2.11 > the freebsd box is running 5.3.2 > > is there a bug in 5.3.2? a search does not turn anything up. > Just curious, what version of FreeBSD? Did you compile PHP & extensions or used the packages? -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] cURL on freebsd
can anyone tell me why the cURL code below produces a POST request as
expected on linux, but the same code on freebsd is forced to be a GET
request?
the linux box is running php 5.2.11
the freebsd box is running 5.3.2
is there a bug in 5.3.2? a search does not turn anything up.
=
function makePostRequest( $url , $params ) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $params);
$headers = array(
'Content-type: application/json',
'Expect:'
);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_UPLOAD, false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
$content = curl_exec($ch);
$errorCode = (int)curl_getinfo($ch,
CURLINFO_HTTP_CODE);
return array("content" => $content, "errorCode" =>
$errorCode);
}
[PHP] curl spider and being a good citizen
I'm writing a custom search engine for my site, it seemed easier than
modifying sphyder (what I currently use) to do what I want especially
since sphyder has a lot of stuff that isn't personally of use to me.
One of the things I want to do when I index is list external links and
check them.
The idea is to have curl download just the headers but not content from
external links.
This is what I have as part of my class to do that -
function meta($url) {
$process = curl_init($url);
curl_setopt($process, CURLOPT_CONNECTTIMEOUT, 15);
curl_setopt($process, CURLOPT_TIMEOUT, 20);
curl_setopt($process, CURLOPT_HTTPHEADER, $this->headers);
curl_setopt($process, CURLOPT_USERAGENT, $this->user_agent);
curl_setopt($process, CURLOPT_NOBODY, true);
curl_setopt($process, CURLOPT_HEADER, 0);
$fetch = curl_exec($process);
$return[] = curl_getinfo($process, CURLINFO_HTTP_CODE);
$meta = split(';',curl_getinfo($process, CURLINFO_CONTENT_TYPE));
$return[] = $meta[0];
curl_close($process);
return $return;
}
I am under the impression that
curl_setopt($process, CURLOPT_NOBODY, true);
does what I want - but the curl docs can be confusing.
Will that work to just download the headers needed to get the http
status code and mime type w/o grabbing content?
And secondly, will making 40 or so connections to the same remote site
just to grab headers but not content (there are two I link to quite a
bit with permission) to check for moved files possibly cause issues with
their server software? It doesn't seem to for me (Apache on Linux) but
that's me, and I'm not positive curl stopped the download after getting
last http header.
Pages on those sites do move as taxonomy moves and the people in charge
don't seem to keep 301 moved redirects in place when they reorganize, so
I do need to check with some frequency, but I don't want to cause problems.
Thanks for suggestions.
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Curl output
On Fri, 2009-10-02 at 18:07 +0530, kranthi wrote: > >> I've read that the upcoming Firefox 4 may have some features built in > >> for this sort of thing, and there are plugins out there for most > >> browsers that can do this as an added layer of security. > Sorry but I could not understand what you meant by "this" > > coming back to original problem... you should keep in mind that if > base tag is used, the links () in that page will become > absolute links instead of relative links > "this" is this: "Some browser security settings may not allow you to run Javascript code that exists on another server though" Also, you won't have to worry about the links, unless you wanted the links that were relative to the remote site to be relative to your own. This would only work out in such a way if you'd scraped their whole site for content, which would beg the question, "why would someone honestly need to scrape an entire site?" Thanks, Ash http://www.ashleysheridan.co.uk -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Curl output
>> I've read that the upcoming Firefox 4 may have some features built in >> for this sort of thing, and there are plugins out there for most >> browsers that can do this as an added layer of security. Sorry but I could not understand what you meant by "this" coming back to original problem... you should keep in mind that if base tag is used, the links () in that page will become absolute links instead of relative links -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Curl output
On Fri, 2009-10-02 at 15:59 +0530, kranthi wrote: > not many users use those kind of browsers, because if they do most of > the websites which use CDNs will not work. I've read that the upcoming Firefox 4 may have some features built in for this sort of thing, and there are plugins out there for most browsers that can do this as an added layer of security. On this front though, has anyone else read about Microsofts Gazelle browser/os? I'm not trying to plug it here or anything (I'm an anti Windows person myself!) but the concept looks interesting. It goes further than Chrome where each tab has it's own process by giving each domain specific object its own process/sandbox. Thanks, Ash http://www.ashleysheridan.co.uk -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Curl output
>> Some browser security settings may not allow you to run Javascript code
>> that exists on another server though
not many users use those kind of browsers, because if they do most of
the websites which use CDNs will not work.
Firstly, it is not a good idea to fetch an entire web page and snow it
to an user. (use iframes if this is a must)
Secondly, this idea may not be feasible if the web page in question
uses AJAX, because none of the browsers allow cross domain AJAX
requests
As a side note, use str_replace("", "",
$text) if base tag doesnot work
On 02/10/2009, Ashley Sheridan wrote:
> On Fri, 2009-10-02 at 12:51 +0530, kranthi wrote:
>> using the > relative..
>> http://www.w3schools.com/TAGS/tag_base.asp
>> but I am not sure if base tag works outside
>>
>> try... before curl_init()
>> where URL is parsed using PHP's parse_url() function on http://example.com
>>
>> in either case if there are any external js/css files cURL wont fetch
>> external files(and if they are internal/inline your browser should
>> display them already).. your browser will have to fetch them for you,
>> net console of firebug will be helpful in that case.
>>
>
> Some browser security settings may not allow you to run Javascript code
> that exists on another server though. The tag can be used to
> reference the external CSS and Javascript, but watch out for security
> settings on client agents.
>
> Thanks,
> Ash
> http://www.ashleysheridan.co.uk
>
>
>
>
--
Kranthi.
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Curl output
On Fri, 2009-10-02 at 12:51 +0530, kranthi wrote: > using the http://www.w3schools.com/TAGS/tag_base.asp > but I am not sure if base tag works outside > > try... before curl_init() > where URL is parsed using PHP's parse_url() function on http://example.com > > in either case if there are any external js/css files cURL wont fetch > external files(and if they are internal/inline your browser should > display them already).. your browser will have to fetch them for you, > net console of firebug will be helpful in that case. > Some browser security settings may not allow you to run Javascript code that exists on another server though. The tag can be used to reference the external CSS and Javascript, but watch out for security settings on client agents. Thanks, Ash http://www.ashleysheridan.co.uk -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Curl output
using the http://www.w3schools.com/TAGS/tag_base.asp but I am not sure if base tag works outside try... before curl_init() where URL is parsed using PHP's parse_url() function on http://example.com in either case if there are any external js/css files cURL wont fetch external files(and if they are internal/inline your browser should display them already).. your browser will have to fetch them for you, net console of firebug will be helpful in that case. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Curl output
gbhumphrey wrote: Hi, I am doing a basical curl call and can get the webpage I want. However when it prints to the screen, it only prints the text, not css or any javascript calls that run on page load. Is there a way to make it do that? thanks using a basic curl call $curl_handle=curl_init(); curl_setopt($curl_handle,CURLOPT_URL,'http://example.com'); curl_setopt($ch, CURLOPT_RETURNTRANSFER,0); // return into a variable curl_exec($curl_handle); Along the lines of what Paul has already alluded to, you will probably find upon further examination of the html source that the "linked" files (js, css, etc...) are probably using relative paths, not absolute paths including the domain name. To fix this, you would have to do a little source code rewriting to add the full domain name and directory path information to the "linked" in url structure. I know if you do this with wget (cli app) that you can have it re-write the domain + path url references. But, I'm not sure if cURL has an option for that. You might RTM on the cURL options page to find out. http://php.net/curl -- Jim Lucas "Some men are born to greatness, some achieve greatness, and some have greatness thrust upon them." Twelfth Night, Act II, Scene V by William Shakespeare -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Curl output
On Thu, Oct 01, 2009 at 04:37:14PM -0700, gbhumphrey wrote: > > Hi, I am doing a basical curl call and can get the webpage I want. However > when it prints to the screen, it only prints the text, not css or any > javascript calls that run on page load. > Is there a way to make it do that? > > thanks > using a basic curl call > $curl_handle=curl_init(); > curl_setopt($curl_handle,CURLOPT_URL,'http://example.com'); > curl_setopt($ch, CURLOPT_RETURNTRANSFER,0); // return into a variable > curl_exec($curl_handle); I don't know about the javascript, but if the CSS is an external file, then it wouldn't get dragged along with the original file, and thus wouldn't be there to style anything on the HTML page. Just a guess. Paul -- Paul M. Foster -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] Curl output
Hi, I am doing a basical curl call and can get the webpage I want. However when it prints to the screen, it only prints the text, not css or any javascript calls that run on page load. Is there a way to make it do that? thanks using a basic curl call $curl_handle=curl_init(); curl_setopt($curl_handle,CURLOPT_URL,'http://example.com'); curl_setopt($ch, CURLOPT_RETURNTRANSFER,0); // return into a variable curl_exec($curl_handle); -- View this message in context: http://www.nabble.com/Curl-output-tp25708385p25708385.html Sent from the PHP - General mailing list archive at Nabble.com.
Re: [PHP] CURL intermittant problem
It looks like a connection problem, You has to make sure php can connect to the IP You can remove the IP from the source, just add an entry in the /etc/hosts file Mrtn On Thu, Jun 11, 2009 at 8:14 PM, Daniel Brown wrote: > On Thu, Jun 11, 2009 at 19:07, > ioan...@btinternet.com wrote: > > > > The target URL works perfectly and quicly if put directly into the > browser > > URL line. > > > > Do you know of any other reasons for problems with such a connection? > > Is the script being run located on the same system as which you're > using the browser to (successfully) connect to the target host? If > not, check to see if there's a network issue - a downed or improperly > bound interface, SELinux restrictions, blacklisted by the remote > machine, etc. - on the server on which the script it hosted. In > addition, as simple and obvious as it is, if it's a private network, > make sure the machine on which the script is hosted is within the same > network as the target machine. > > -- > > daniel.br...@parasane.net || danbr...@php.net > http://www.parasane.net/ || http://www.pilotpig.net/ > 50% Off All Shared Hosting Plans at PilotPig: Use Coupon DOW1 > > -- > PHP General Mailing List (http://www.php.net/) > To unsubscribe, visit: http://www.php.net/unsub.php > > -- Martin Scotta
Re: [PHP] CURL intermittant problem
On Thu, Jun 11, 2009 at 19:07, ioan...@btinternet.com wrote: > > The target URL works perfectly and quicly if put directly into the browser > URL line. > > Do you know of any other reasons for problems with such a connection? Is the script being run located on the same system as which you're using the browser to (successfully) connect to the target host? If not, check to see if there's a network issue - a downed or improperly bound interface, SELinux restrictions, blacklisted by the remote machine, etc. - on the server on which the script it hosted. In addition, as simple and obvious as it is, if it's a private network, make sure the machine on which the script is hosted is within the same network as the target machine. -- daniel.br...@parasane.net || danbr...@php.net http://www.parasane.net/ || http://www.pilotpig.net/ 50% Off All Shared Hosting Plans at PilotPig: Use Coupon DOW1 -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] CURL intermittant problem
I have been having problems with a curl script which works normally with many different URLs but had a particular intermittant problem with a url in the following format: http://10.20.30.40/0001/032/023112/filename.phtml?param1=paramvalue1¶m2=paramvalue2 etc etc. The unusual thing about this URL is that is starts with an IP address. I got a lot of failures to connect, so tried a different way to connect as all I need is the page as a string: According to php.net, the function 'file_get_contents()' should work if allow_url_fopen is set to TRUE for the PHP installation. 'allow_url_fopen' is set to 1 in my php.ini file. http://php.net/file_get_contents http://php.net/allow_url_fopen However, 'file_get_contents' also fails to connect to this particular URL. It gives: "failed to open stream: Connection timed out in */home/shortsta/public_html/live48_test.php* on line *75* *Warning*: feof(): supplied argument is not a valid stream resource in */home/mysite/public_html/testpage.php* on line *79*" The target URL works perfectly and quicly if put directly into the browser URL line. Do you know of any other reasons for problems with such a connection? John -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] cURL loop?
On Thu, May 28, 2009 at 23:31, espontaneo wrote: > > Hello! I am currently working on a script that will scrape data from a > property advertising web page. The web page has multiple pages. What I'm > getting is only the first page. What I wanted to do is to use curl to scrape > all the data from that page. I just learned php so I don't know how I can do > this. There are a variety of ways, but rather than writing your own spider script, you may want to look into the built-in spidering capabilities of `wget` and `curl` from the command line. Both have Windows and *NIX builds, so platform isn't an issue; so long as you have access to the shell, you should be fine. -- daniel.br...@parasane.net || danbr...@php.net http://www.parasane.net/ || http://www.pilotpig.net/ 50% Off All Shared Hosting Plans at PilotPig: Use Coupon DOW1 -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] cURL loop?
Hello! I am currently working on a script that will scrape data from a property advertising web page. The web page has multiple pages. What I'm getting is only the first page. What I wanted to do is to use curl to scrape all the data from that page. I just learned php so I don't know how I can do this. -- View this message in context: http://www.nabble.com/cURL-loop--tp23773748p23773748.html Sent from the PHP - General mailing list archive at Nabble.com. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
RES: RES: [PHP] CURL problems still
Hi Terion,
I thing the problems is the time limit on your Server.
Try this now:
Test the var before insert into db:
If(!isset($crime)){$crime=''}
After u remove from the db the unwanted lines.
On the end of the file u do this:
$letters=array();
for($i='A';$i<='Z';$i++){$letters[$i]=$i;if($i=='Z'){break;}}
$tL=count($letters);
$letters=array_keys($letters);
if(isset($_GET['L'])){if(trim($_GET['L'])!==''){if($_GET['L']+1>$tL){$next=0
;$final=1;}else{$next=$_GET['L'];}}else{$next=0;}}else{$next=0;}
$nextLetter=$letters[$next];
if(!isset($final)){
//do the task
//on the end of the file u put this:
for($i=0;$i<$tL;$i++){if($nextLetter==$letters[$i]){if($i+1==$tL){$nextLette
r=0;$file='A';}else{$nextLetter=$i+1;$file=$letters[$i+1];}break;}}
#header('Location:'.$file.'.php?L='.$nextLetter);
}
U'll need to make A.php, B.php, C.php, D.php, etc... file, with the same
content., did u get it?
Zechim
-Mensagem original-
De: Miller, Terion [mailto:tmil...@springfi.gannett.com]
Enviada em: terça-feira, 12 de maio de 2009 10:24
Para: Phpster; Miller, Terion
Cc: Jônatas Zechim; PHP-General List
Assunto: Re: RES: [PHP] CURL problems still
>
> Bastien, could you give me an example of what you mean?
> would it be $Name = ""
>
> like that?
>
> Thanks for the ideas everyone!
>
>> PHP General Mailing List (http://www.php.net/)
>> To unsubscribe, visit: http://www.php.net/unsub.php
>>
>
> Try Imitliazing the variables at the top of the loop to empty strings
> or zeros
>
> Bastien
>
>
Yep, that's it
$_thing = '';
Bastien
Well I tried it and still it stopped at the "C's" maybe I put them in the
wrong spot? I tried something yesterday where I commented out the "C's"
url, and it ran through the "F's" could a setting somewhere be restricting
how many files it can run...I am just stumped, the script runs fine, inserts
the data I'm wanting etc, just can't get it to run all 26 pagesargh.
Code--
foreach($html->find('table') as $table) {foreach($table->find('tr') as
$tr){ //set variables to nada ? $name = ''; $age = '';
$warrant = ''; $bond = ''; $wnumber = ''; $crime = '';
// Grab children $cells = $tr->children();
if($cells[0]->plaintext != "Name"){for ($i = 0;
$i < count($cells); $i++){
switch ($i){ case 0: // Name $name =
$cells[$i]->plaintext; echo $cells[$i]->plaintext;
break; case 1: // Age $age =
$cells[$i]->plaintext; break; case 2: //
Warrant type $warrant = $cells[$i]->plaintext;
break; case 3: // Bond amount $bond =
$cells[$i]->plaintext; break; case 4: //
Warrant number $wnumber = $cells[$i]->plaintext;
break; case 5: // Offence description
$crime = $cells[$i]->plaintext; break;
Default: echo "Uh-oh"; }
} }// Build your
INSERT statement here $query = "INSERT into `warrants` (name,
age, warrant, bond, wnumber, crime) VALUES (";$query .= " '$name',
'$age', '$warrant', '$bond', '$wnumber', '$crime' )";$wid =
mysql_insert_id();echo $query;// run query
mysql_query($query) or die (mysql_error());
-end code---
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: RES: [PHP] CURL problems still
Nathan Rixham wrote:
Miller, Terion wrote:
Well I tried it and still it stopped at the "C's"
1:
your script is timing out, you need to set the time limit higher
set_time_limit(0);
2:
foreach($html->find('table') as $table) {
$rows = explode('' , $table );
$headerCells = explode('', array_shift($rows));
$struct = array();
foreach( $headerCells as $c => $cell ) {
$struct[] = str_replace(' ', '_', trim(strtolower(strip_tags( $cell ;
}
$data = array();
foreach( $rows as $r => $row ) {
$cells = explode( '' , $row );
$item = array();
foreach( $cells as $c => $cell ) {
$item[$struct[$c]] = trim(strip_tags( $cell ));
}
$data[] = $item;
$query = "INSERT into `warrants` ";
$query .= "(name, age, warrant, bond, wnumber, crime)";
$query .= " VALUES (";
$query .= " '{$item['name']}',";
$query .= " '{$item['age']}',";
$query .= " '{$item['warrant_type']}',";
$query .= " '{$item['bond']}',";
$query .= " '{$item['warrant_number']}',";
$query .= " '{$item['offence_description']}'";
$query .= " )";
echo $query . "\n";
// run query here
}
}
3:
If this is your own site then don't read on, but if it's not your own
site and this is a paid job for a client then..
do you not think there is something a bit wrong about taking on work
which you obviously can't do, telling the client you can, then
continually asking hard working people to take time out from their own
working days to both teach you and complete your work for you?
There is such an increase in people just saying "I know PHP" taking on
work then getting the helpful people on this list and others to do it
for them, and it really really sucks. If you can't do the work don't
take it on, if you get genuinely stuck then ask - but don't just copy
and paste a clients job description here with some random code and
expect others to do it for you.
In the time you've spent asking and waiting, you could have learnt how
to do it yourself.
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: RES: [PHP] CURL problems still
Miller, Terion wrote:
Well I tried it and still it stopped at the "C's"
1:
your script is timing out, you need to set the time limit higher
set_time_limit(0);
2:
foreach($html->find('table') as $table) {
$rows = explode('' , $table );
$headerCells = explode('', array_shift($rows));
$struct = array();
foreach( $headerCells as $c => $cell ) {
$struct[] = str_replace(' ', '_', trim(strtolower(strip_tags( $cell ;
}
$data = array();
foreach( $rows as $r => $row ) {
$cells = explode( '' , $row );
$item = array();
foreach( $cells as $c => $cell ) {
$item[$struct[$c]] = trim(strip_tags( $cell ));
}
$data[] = $item;
$query = "INSERT into `warrants` ";
$query .= "(name, age, warrant, bond, wnumber, crime)";
$query .= " VALUES (";
$query .= " '{$item['name']}',";
$query .= " '{$item['age']}',";
$query .= " '{$item['warrant_type']}',";
$query .= " '{$item['bond']}',";
$query .= " '{$item['warrant_number']}',";
$query .= " '{$item['offence_description']}'";
$query .= " )";
echo $query . "\n";
// run query here
}
}
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: RES: [PHP] CURL problems still
>
> Bastien, could you give me an example of what you mean?
> would it be $Name = ""
>
> like that?
>
> Thanks for the ideas everyone!
>
>> PHP General Mailing List (http://www.php.net/)
>> To unsubscribe, visit: http://www.php.net/unsub.php
>>
>
> Try Imitliazing the variables at the top of the loop to empty strings
> or zeros
>
> Bastien
>
>
Yep, that's it
$_thing = '';
Bastien
Well I tried it and still it stopped at the "C's" maybe I put them in the wrong
spot? I tried something yesterday where I commented out the "C's" url, and it
ran through the "F's" could a setting somewhere be restricting how many files
it can run...I am just stumped, the script runs fine, inserts the data I'm
wanting etc, just can't get it to run all 26 pagesargh.
Code--
foreach($html->find('table') as $table) {foreach($table->find('tr') as $tr)
{ //set variables to nada ? $name = ''; $age = ''; $warrant =
''; $bond = ''; $wnumber = ''; $crime = '';// Grab
children $cells = $tr->children();
if($cells[0]->plaintext != "Name"){for ($i = 0; $i
< count($cells); $i++){ switch
($i){ case 0: // Name $name =
$cells[$i]->plaintext; echo $cells[$i]->plaintext;
break; case 1: // Age $age =
$cells[$i]->plaintext; break; case 2: //
Warrant type $warrant = $cells[$i]->plaintext;
break; case 3: // Bond amount $bond
= $cells[$i]->plaintext; break; case 4: //
Warrant number $wnumber = $cells[$i]->plaintext;
break; case 5: // Offence description
$crime = $cells[$i]->plaintext; break;
Default: echo "Uh-oh"; }
}
}// Build your INSERT statement here
$query = "INSERT into `warrants` (name, age, warrant, bond, wnumber, crime)
VALUES (";$query .= " '$name', '$age', '$warrant', '$bond', '$wnumber',
'$crime' )";$wid = mysql_insert_id();echo $query;
// run query mysql_query($query) or die (mysql_error());
-end code---
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: RES: [PHP] CURL problems still
On May 11, 2009, at 17:53, "Miller, Terion" > wrote: Bastien, could you give me an example of what you mean? would it be $Name = "" like that? Thanks for the ideas everyone! -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php Try Imitliazing the variables at the top of the loop to empty strings or zeros Bastien Yep, that's it $_thing = ''; Bastien -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: RES: [PHP] CURL problems still
Bastien, could you give me an example of what you mean? would it be $Name = "" like that? Thanks for the ideas everyone! > > > > > -- > PHP General Mailing List (http://www.php.net/) > To unsubscribe, visit: http://www.php.net/unsub.php > Try Imitliazing the variables at the top of the loop to empty strings or zeros Bastien -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: RES: [PHP] CURL problems still
On May 11, 2009, at 16:01, "Miller, Terion" > wrote:
The error I get is from my INSERT statement
It does insert the records up to C page though...
The problem area:
// Build your INSERT statement here $query = "INSERT
into warrants (wid, name, age, warrant, bond, wnumber, crime) VALUES
(";$query .= " '$wid', '$name', '$age', '$warrant', '$bond',
'$wnumber', '$crime' )";$wid =
mysql_insert_id();echo $query;// run
query mysql_query($query) or die (mysql_error());
The Error message
Notice: Undefined variable: wid in /var/www/vhosts/getpublished.news-
leader.com/httpdocs/Warrants/loopScrape1.php on line 124
Notice: Undefined variable: name in /var/www/vhosts/
getpublished.news-leader.com/httpdocs/Warrants/loopScrape1.php on
line 124
Notice: Undefined variable: age in /var/www/vhosts/getpublished.news-
leader.com/httpdocs/Warrants/loopScrape1.php on line 124
Notice: Undefined variable: warrant in /var/www/vhosts/
getpublished.news-leader.com/httpdocs/Warrants/loopScrape1.php on
line 124
Notice: Undefined variable: bond in /var/www/vhosts/
getpublished.news-leader.com/httpdocs/Warrants/loopScrape1.php on
line 124
Notice: Undefined variable: wnumber in /var/www/vhosts/
getpublished.news-leader.com/httpdocs/Warrants/loopScrape1.php on
line 124
Notice: Undefined variable: crime in /var/www/vhosts/
getpublished.news-leader.com/httpdocs/Warrants/loopScrape1.php on
line 124
On 5/11/09 2:45 PM, "Jônatas Zechim" wrote:
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Try Imitliazing the variables at the top of the loop to empty strings
or zeros
Bastien
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: RES: [PHP] CURL problems still
The error I get is from my INSERT statement
It does insert the records up to C page though...
The problem area:
// Build your INSERT statement here $query = "INSERT into
warrants (wid, name, age, warrant, bond, wnumber, crime) VALUES (";$query
.= " '$wid', '$name', '$age', '$warrant', '$bond', '$wnumber', '$crime' )";
$wid = mysql_insert_id();echo $query;// run
query mysql_query($query) or die (mysql_error());
The Error message
Notice: Undefined variable: wid in
/var/www/vhosts/getpublished.news-leader.com/httpdocs/Warrants/loopScrape1.php
on line 124
Notice: Undefined variable: name in
/var/www/vhosts/getpublished.news-leader.com/httpdocs/Warrants/loopScrape1.php
on line 124
Notice: Undefined variable: age in
/var/www/vhosts/getpublished.news-leader.com/httpdocs/Warrants/loopScrape1.php
on line 124
Notice: Undefined variable: warrant in
/var/www/vhosts/getpublished.news-leader.com/httpdocs/Warrants/loopScrape1.php
on line 124
Notice: Undefined variable: bond in
/var/www/vhosts/getpublished.news-leader.com/httpdocs/Warrants/loopScrape1.php
on line 124
Notice: Undefined variable: wnumber in
/var/www/vhosts/getpublished.news-leader.com/httpdocs/Warrants/loopScrape1.php
on line 124
Notice: Undefined variable: crime in
/var/www/vhosts/getpublished.news-leader.com/httpdocs/Warrants/loopScrape1.php
on line 124
On 5/11/09 2:45 PM, "Jônatas Zechim" wrote:
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
RES: [PHP] CURL problems still
Hey Guy, try to change the 'max_execution_time'
ini_set('max_execution_time','1800');
-Mensagem original-
De: Miller, Terion [mailto:tmil...@springfi.gannett.com]
Enviada em: segunda-feira, 11 de maio de 2009 16:41
Para: PHP-General List
Assunto: [PHP] CURL problems still
I'm back. I thought I had this script working but it seems to always stop
at the end of the C's, so I can get it to go thru pages A-C then it stops, I
have tried setting the time-out to a very large amount but it never
continues to run past the same spot, it gathers all records to the end of C,
then quits...any ideas?
Script:
http://www.greenecountymo.org/sheriff/warrants.php?search=A";;
$targets["b"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=B";;
$targets["c"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=C";;
$targets["d"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=D";;
$targets["e"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=E";;
$targets["f"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=F";;
$targets["g"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=G";;
$targets["h"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=H";;
$targets["i"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=I";;
$targets["j"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=J";;
$targets["k"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=K";;
$targets["l"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=L";;
$targets["m"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=M";;
$targets["n"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=N";;
$targets["o"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=O";;
$targets["p"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=P";;
$targets["q"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=Q";;
$targets["r"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=R";;
$targets["s"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=S";;
$targets["t"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=T";;
$targets["u"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=U";;
$targets["v"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=V";;
$targets["w"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=W";;
$targets["x"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=X";;
$targets["y"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=Y";;
$targets["z"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=Z";;
foreach
($targets as $target_url){
echo $target_url;
$userAgent = 'Googlebot/2.1
(http://www.googlebot.com/bot.html)';
$ch = curl_init();
curl_setopt($ch,
CURLOPT_USERAGENT, $userAgent);
curl_setopt($ch,
CURLOPT_URL,$target_url);
curl_setopt($ch, CURLOPT_FAILONERROR,
true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, false);
curl_setopt($ch,
CURLOPT_AUTOREFERER, true);
curl_setopt($ch,
CURLOPT_RETURNTRANSFER,true);
curl_setopt($ch, CURLOPT_TIMEOUT,
6000);
$html = curl_exec($ch);
if (!$html) {
echo "cURL error
number:" .curl_errno($ch);
echo "cURL error:" . curl_error($ch);
exit;
}
// Create DOM from URL or file
$html =
file_get_html($target_url);
// Find table
foreach($html->find('table') as
$table)
{
foreach($table->find('tr') as $tr)
{
// Grab children
$cells = $tr->children();
if($cells[0]->plaintext != "Name")
{
for ($i =
0; $i < count($cells); $i++)
{
switch ($i){
case 0: // Name
$name =
$cells[$i]->plaintext;
echo $cells[$i]->plaintext;
break;
case 1: // Age
$age =
$cells[$i]->plaintext;
break;
case 2:
// Warrant type
$warrant = $cells[$i]->plaintext;
break;
case 3: // Bond amount
$bond =
$cells[$i]->plaintext;
break;
case 4:
// Warrant number
$wnumber = $cells[$i]->plaintext;
break;
case 5: // Offen
[PHP] CURL problems still
I'm back. I thought I had this script working but it seems to always stop
at the end of the C's, so I can get it to go thru pages A-C then it stops, I
have tried setting the time-out to a very large amount but it never
continues to run past the same spot, it gathers all records to the end of C,
then quits...any ideas?
Script:
http://www.greenecountymo.org/sheriff/warrants.php?search=A";;
$targets["b"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=B";;
$targets["c"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=C";;
$targets["d"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=D";;
$targets["e"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=E";;
$targets["f"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=F";;
$targets["g"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=G";;
$targets["h"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=H";;
$targets["i"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=I";;
$targets["j"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=J";;
$targets["k"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=K";;
$targets["l"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=L";;
$targets["m"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=M";;
$targets["n"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=N";;
$targets["o"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=O";;
$targets["p"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=P";;
$targets["q"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=Q";;
$targets["r"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=R";;
$targets["s"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=S";;
$targets["t"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=T";;
$targets["u"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=U";;
$targets["v"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=V";;
$targets["w"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=W";;
$targets["x"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=X";;
$targets["y"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=Y";;
$targets["z"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=Z";;
foreach
($targets as $target_url){
echo $target_url;
$userAgent = 'Googlebot/2.1
(http://www.googlebot.com/bot.html)';
$ch = curl_init();
curl_setopt($ch,
CURLOPT_USERAGENT, $userAgent);
curl_setopt($ch,
CURLOPT_URL,$target_url);
curl_setopt($ch, CURLOPT_FAILONERROR,
true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, false);
curl_setopt($ch,
CURLOPT_AUTOREFERER, true);
curl_setopt($ch,
CURLOPT_RETURNTRANSFER,true);
curl_setopt($ch, CURLOPT_TIMEOUT,
6000);
$html = curl_exec($ch);
if (!$html) {
echo "cURL error
number:" .curl_errno($ch);
echo "cURL error:" . curl_error($ch);
exit;
}
// Create DOM from URL or file
$html =
file_get_html($target_url);
// Find table
foreach($html->find('table') as
$table)
{
foreach($table->find('tr') as $tr)
{
// Grab children
$cells = $tr->children();
if($cells[0]->plaintext != "Name")
{
for ($i =
0; $i < count($cells); $i++)
{
switch ($i){
case 0: // Name
$name =
$cells[$i]->plaintext;
echo $cells[$i]->plaintext;
break;
case 1: // Age
$age =
$cells[$i]->plaintext;
break;
case 2:
// Warrant type
$warrant = $cells[$i]->plaintext;
break;
case 3: // Bond amount
$bond =
$cells[$i]->plaintext;
break;
case 4:
// Warrant number
$wnumber = $cells[$i]->plaintext;
break;
case 5: // Offence description
$crime = $cells[$i]->plaintext;
break;
Default:
echo "Uh-oh";
}
}
}
// Build your INSERT statement here
// Build your INSERT statement here
$query = "INSERT
into warrants (wid, name, age, warrant, bond, wnumber, crime) VALUES (";
$query .= " '$wid', '$name', '$age', '$warrant', '$bond', '$wnumber',
'$crime' )";
//$wid = mysql_insert_id();
echo
$query;
// run query
mysql_query($query) or die
(mysql_error());
}
}
}
?>
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
RES: RES: RES: RES: [PHP] CURL error help
U r correct! It's ok know.
-Mensagem original-
De: Shawn McKenzie [mailto:nos...@mckenzies.net]
Enviada em: sexta-feira, 8 de maio de 2009 13:34
Para: Miller, Terion
Cc: Jônatas Zechim; PHP-General List
Assunto: Re: RES: RES: RES: [PHP] CURL error help
Miller, Terion wrote:
>
>
> On 5/8/09 10:55 AM, "Jônatas Zechim" wrote:
>
> Ok, but u need to confirm the $args vars are being passed to that
funtion,
> because I thing they're blanks.
>
> Zechim
> zechim.com
> São Paulo/Brazil
>
> here is the code from the dom.php file, as it stands currently my error
reads:
>
>
> Warning: file_get_contents($target_url) [function.file-get-contents]:
failed to open stream: No such file or directory in
/var/www/vhosts/getpublished.news-leader.com/httpdocs/Warrants/inc/dom.php
on line 39
>
> Here is the code from dom.php:
>
>
> define('HDOM_TYPE_ELEMENT', 1);
> define('HDOM_TYPE_COMMENT', 2);
> define('HDOM_TYPE_TEXT',3);
> define('HDOM_TYPE_ENDTAG', 4);
> define('HDOM_TYPE_ROOT',5);
> define('HDOM_TYPE_UNKNOWN', 6);
> define('HDOM_QUOTE_DOUBLE', 0);
> define('HDOM_QUOTE_SINGLE', 1);
> define('HDOM_QUOTE_NO', 3);
> define('HDOM_INFO_BEGIN', 0);
> define('HDOM_INFO_END', 1);
> define('HDOM_INFO_QUOTE', 2);
> define('HDOM_INFO_SPACE', 3);
> define('HDOM_INFO_TEXT',4);
> define('HDOM_INFO_INNER', 5);
> define('HDOM_INFO_OUTER', 6);
> define('HDOM_INFO_ENDSPACE',7);
>
> // helper functions
> //
-
> // get html dom from file
> function file_get_html() {
>$dom = new simple_html_dom;
>$args = func_get_args();
>$dom->load(call_user_func_array('file_get_contents', $args), true);
>return $dom;
>
> }
>
> // get html dom from string
> function str_get_html($str, $lowercase=true) {
>$dom = new simple_html_dom;
>$dom->load($str, $lowercase);
>return $dom;
> }
>
> // dump html dom tree
> function dump_html_tree($node, $show_attr=true, $deep=0) {
>$lead = str_repeat('', $deep);
>echo $lead.$node->tag;
>if ($show_attr && count($node->attr)>0) {
>echo '(';
>foreach($node->attr as $k=>$v)
>echo "[$k]=>\"".$node->$k.'", ';
>echo ')';
>}
>echo "\n";
>
>foreach($node->nodes as $c)
>dump_html_tree($c, $show_attr, $deep+1);
> }
>
> // get dom form file (deprecated)
> function file_get_dom() {
>$dom = new simple_html_dom;
>$args = func_get_args();
>$dom->load(call_user_func_array('file_get_contents', $args), true);
>return $dom;
> }
>
> // get dom form string (deprecated)
> function str_get_dom($str, $lowercase=true) {
>$dom = new simple_html_dom;
>$dom->load($str, $lowercase);
>return $dom;
> }
>
>
I've answered your question. It's not in that file it's in your
function call to file_get_html(). You've got your parameter variable in
single frigging quotes!
--
Thanks!
-Shawn
http://www.spidean.com
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
