RES: RES: [PHP] CURL problems still
Hi Terion,
I thing the problems is the time limit on your Server.
Try this now:
Test the var before insert into db:
If(!isset($crime)){$crime=''}
After u remove from the db the unwanted lines.
On the end of the file u do this:
$letters=array();
for($i='A';$i<='Z';$i++){$letters[$i]=$i;if($i=='Z'){break;}}
$tL=count($letters);
$letters=array_keys($letters);
if(isset($_GET['L'])){if(trim($_GET['L'])!==''){if($_GET['L']+1>$tL){$next=0
;$final=1;}else{$next=$_GET['L'];}}else{$next=0;}}else{$next=0;}
$nextLetter=$letters[$next];
if(!isset($final)){
//do the task
//on the end of the file u put this:
for($i=0;$i<$tL;$i++){if($nextLetter==$letters[$i]){if($i+1==$tL){$nextLette
r=0;$file='A';}else{$nextLetter=$i+1;$file=$letters[$i+1];}break;}}
#header('Location:'.$file.'.php?L='.$nextLetter);
}
U'll need to make A.php, B.php, C.php, D.php, etc... file, with the same
content., did u get it?
Zechim
-Mensagem original-
De: Miller, Terion [mailto:tmil...@springfi.gannett.com]
Enviada em: terça-feira, 12 de maio de 2009 10:24
Para: Phpster; Miller, Terion
Cc: Jônatas Zechim; PHP-General List
Assunto: Re: RES: [PHP] CURL problems still
>
> Bastien, could you give me an example of what you mean?
> would it be $Name = ""
>
> like that?
>
> Thanks for the ideas everyone!
>
>> PHP General Mailing List (http://www.php.net/)
>> To unsubscribe, visit: http://www.php.net/unsub.php
>>
>
> Try Imitliazing the variables at the top of the loop to empty strings
> or zeros
>
> Bastien
>
>
Yep, that's it
$_thing = '';
Bastien
Well I tried it and still it stopped at the "C's" maybe I put them in the
wrong spot? I tried something yesterday where I commented out the "C's"
url, and it ran through the "F's" could a setting somewhere be restricting
how many files it can run...I am just stumped, the script runs fine, inserts
the data I'm wanting etc, just can't get it to run all 26 pagesargh.
Code--
foreach($html->find('table') as $table) {foreach($table->find('tr') as
$tr){ //set variables to nada ? $name = ''; $age = '';
$warrant = ''; $bond = ''; $wnumber = ''; $crime = '';
// Grab children $cells = $tr->children();
if($cells[0]->plaintext != "Name"){for ($i = 0;
$i < count($cells); $i++){
switch ($i){ case 0: // Name $name =
$cells[$i]->plaintext; echo $cells[$i]->plaintext;
break; case 1: // Age $age =
$cells[$i]->plaintext; break; case 2: //
Warrant type $warrant = $cells[$i]->plaintext;
break; case 3: // Bond amount $bond =
$cells[$i]->plaintext; break; case 4: //
Warrant number $wnumber = $cells[$i]->plaintext;
break; case 5: // Offence description
$crime = $cells[$i]->plaintext; break;
Default: echo "Uh-oh"; }
} }// Build your
INSERT statement here $query = "INSERT into `warrants` (name,
age, warrant, bond, wnumber, crime) VALUES (";$query .= " '$name',
'$age', '$warrant', '$bond', '$wnumber', '$crime' )";$wid =
mysql_insert_id();echo $query;// run query
mysql_query($query) or die (mysql_error());
-end code---
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: RES: [PHP] CURL problems still
Nathan Rixham wrote:
Miller, Terion wrote:
Well I tried it and still it stopped at the "C's"
1:
your script is timing out, you need to set the time limit higher
set_time_limit(0);
2:
foreach($html->find('table') as $table) {
$rows = explode('' , $table );
$headerCells = explode('', array_shift($rows));
$struct = array();
foreach( $headerCells as $c => $cell ) {
$struct[] = str_replace(' ', '_', trim(strtolower(strip_tags( $cell ;
}
$data = array();
foreach( $rows as $r => $row ) {
$cells = explode( '' , $row );
$item = array();
foreach( $cells as $c => $cell ) {
$item[$struct[$c]] = trim(strip_tags( $cell ));
}
$data[] = $item;
$query = "INSERT into `warrants` ";
$query .= "(name, age, warrant, bond, wnumber, crime)";
$query .= " VALUES (";
$query .= " '{$item['name']}',";
$query .= " '{$item['age']}',";
$query .= " '{$item['warrant_type']}',";
$query .= " '{$item['bond']}',";
$query .= " '{$item['warrant_number']}',";
$query .= " '{$item['offence_description']}'";
$query .= " )";
echo $query . "\n";
// run query here
}
}
3:
If this is your own site then don't read on, but if it's not your own
site and this is a paid job for a client then..
do you not think there is something a bit wrong about taking on work
which you obviously can't do, telling the client you can, then
continually asking hard working people to take time out from their own
working days to both teach you and complete your work for you?
There is such an increase in people just saying "I know PHP" taking on
work then getting the helpful people on this list and others to do it
for them, and it really really sucks. If you can't do the work don't
take it on, if you get genuinely stuck then ask - but don't just copy
and paste a clients job description here with some random code and
expect others to do it for you.
In the time you've spent asking and waiting, you could have learnt how
to do it yourself.
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: RES: [PHP] CURL problems still
Miller, Terion wrote:
Well I tried it and still it stopped at the "C's"
1:
your script is timing out, you need to set the time limit higher
set_time_limit(0);
2:
foreach($html->find('table') as $table) {
$rows = explode('' , $table );
$headerCells = explode('', array_shift($rows));
$struct = array();
foreach( $headerCells as $c => $cell ) {
$struct[] = str_replace(' ', '_', trim(strtolower(strip_tags( $cell ;
}
$data = array();
foreach( $rows as $r => $row ) {
$cells = explode( '' , $row );
$item = array();
foreach( $cells as $c => $cell ) {
$item[$struct[$c]] = trim(strip_tags( $cell ));
}
$data[] = $item;
$query = "INSERT into `warrants` ";
$query .= "(name, age, warrant, bond, wnumber, crime)";
$query .= " VALUES (";
$query .= " '{$item['name']}',";
$query .= " '{$item['age']}',";
$query .= " '{$item['warrant_type']}',";
$query .= " '{$item['bond']}',";
$query .= " '{$item['warrant_number']}',";
$query .= " '{$item['offence_description']}'";
$query .= " )";
echo $query . "\n";
// run query here
}
}
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: RES: [PHP] CURL problems still
>
> Bastien, could you give me an example of what you mean?
> would it be $Name = ""
>
> like that?
>
> Thanks for the ideas everyone!
>
>> PHP General Mailing List (http://www.php.net/)
>> To unsubscribe, visit: http://www.php.net/unsub.php
>>
>
> Try Imitliazing the variables at the top of the loop to empty strings
> or zeros
>
> Bastien
>
>
Yep, that's it
$_thing = '';
Bastien
Well I tried it and still it stopped at the "C's" maybe I put them in the wrong
spot? I tried something yesterday where I commented out the "C's" url, and it
ran through the "F's" could a setting somewhere be restricting how many files
it can run...I am just stumped, the script runs fine, inserts the data I'm
wanting etc, just can't get it to run all 26 pagesargh.
Code--
foreach($html->find('table') as $table) {foreach($table->find('tr') as $tr)
{ //set variables to nada ? $name = ''; $age = ''; $warrant =
''; $bond = ''; $wnumber = ''; $crime = '';// Grab
children $cells = $tr->children();
if($cells[0]->plaintext != "Name"){for ($i = 0; $i
< count($cells); $i++){ switch
($i){ case 0: // Name $name =
$cells[$i]->plaintext; echo $cells[$i]->plaintext;
break; case 1: // Age $age =
$cells[$i]->plaintext; break; case 2: //
Warrant type $warrant = $cells[$i]->plaintext;
break; case 3: // Bond amount $bond
= $cells[$i]->plaintext; break; case 4: //
Warrant number $wnumber = $cells[$i]->plaintext;
break; case 5: // Offence description
$crime = $cells[$i]->plaintext; break;
Default: echo "Uh-oh"; }
}
}// Build your INSERT statement here
$query = "INSERT into `warrants` (name, age, warrant, bond, wnumber, crime)
VALUES (";$query .= " '$name', '$age', '$warrant', '$bond', '$wnumber',
'$crime' )";$wid = mysql_insert_id();echo $query;
// run query mysql_query($query) or die (mysql_error());
-end code---
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: RES: [PHP] CURL problems still
On May 11, 2009, at 17:53, "Miller, Terion" > wrote: Bastien, could you give me an example of what you mean? would it be $Name = "" like that? Thanks for the ideas everyone! -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php Try Imitliazing the variables at the top of the loop to empty strings or zeros Bastien Yep, that's it $_thing = ''; Bastien -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: RES: [PHP] CURL problems still
Bastien, could you give me an example of what you mean? would it be $Name = "" like that? Thanks for the ideas everyone! > > > > > -- > PHP General Mailing List (http://www.php.net/) > To unsubscribe, visit: http://www.php.net/unsub.php > Try Imitliazing the variables at the top of the loop to empty strings or zeros Bastien -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: RES: [PHP] CURL problems still
On May 11, 2009, at 16:01, "Miller, Terion" > wrote:
The error I get is from my INSERT statement
It does insert the records up to C page though...
The problem area:
// Build your INSERT statement here $query = "INSERT
into warrants (wid, name, age, warrant, bond, wnumber, crime) VALUES
(";$query .= " '$wid', '$name', '$age', '$warrant', '$bond',
'$wnumber', '$crime' )";$wid =
mysql_insert_id();echo $query;// run
query mysql_query($query) or die (mysql_error());
The Error message
Notice: Undefined variable: wid in /var/www/vhosts/getpublished.news-
leader.com/httpdocs/Warrants/loopScrape1.php on line 124
Notice: Undefined variable: name in /var/www/vhosts/
getpublished.news-leader.com/httpdocs/Warrants/loopScrape1.php on
line 124
Notice: Undefined variable: age in /var/www/vhosts/getpublished.news-
leader.com/httpdocs/Warrants/loopScrape1.php on line 124
Notice: Undefined variable: warrant in /var/www/vhosts/
getpublished.news-leader.com/httpdocs/Warrants/loopScrape1.php on
line 124
Notice: Undefined variable: bond in /var/www/vhosts/
getpublished.news-leader.com/httpdocs/Warrants/loopScrape1.php on
line 124
Notice: Undefined variable: wnumber in /var/www/vhosts/
getpublished.news-leader.com/httpdocs/Warrants/loopScrape1.php on
line 124
Notice: Undefined variable: crime in /var/www/vhosts/
getpublished.news-leader.com/httpdocs/Warrants/loopScrape1.php on
line 124
On 5/11/09 2:45 PM, "Jônatas Zechim" wrote:
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Try Imitliazing the variables at the top of the loop to empty strings
or zeros
Bastien
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: RES: [PHP] CURL problems still
The error I get is from my INSERT statement
It does insert the records up to C page though...
The problem area:
// Build your INSERT statement here $query = "INSERT into
warrants (wid, name, age, warrant, bond, wnumber, crime) VALUES (";$query
.= " '$wid', '$name', '$age', '$warrant', '$bond', '$wnumber', '$crime' )";
$wid = mysql_insert_id();echo $query;// run
query mysql_query($query) or die (mysql_error());
The Error message
Notice: Undefined variable: wid in
/var/www/vhosts/getpublished.news-leader.com/httpdocs/Warrants/loopScrape1.php
on line 124
Notice: Undefined variable: name in
/var/www/vhosts/getpublished.news-leader.com/httpdocs/Warrants/loopScrape1.php
on line 124
Notice: Undefined variable: age in
/var/www/vhosts/getpublished.news-leader.com/httpdocs/Warrants/loopScrape1.php
on line 124
Notice: Undefined variable: warrant in
/var/www/vhosts/getpublished.news-leader.com/httpdocs/Warrants/loopScrape1.php
on line 124
Notice: Undefined variable: bond in
/var/www/vhosts/getpublished.news-leader.com/httpdocs/Warrants/loopScrape1.php
on line 124
Notice: Undefined variable: wnumber in
/var/www/vhosts/getpublished.news-leader.com/httpdocs/Warrants/loopScrape1.php
on line 124
Notice: Undefined variable: crime in
/var/www/vhosts/getpublished.news-leader.com/httpdocs/Warrants/loopScrape1.php
on line 124
On 5/11/09 2:45 PM, "Jônatas Zechim" wrote:
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
RES: [PHP] CURL problems still
Hey Guy, try to change the 'max_execution_time'
ini_set('max_execution_time','1800');
-Mensagem original-
De: Miller, Terion [mailto:tmil...@springfi.gannett.com]
Enviada em: segunda-feira, 11 de maio de 2009 16:41
Para: PHP-General List
Assunto: [PHP] CURL problems still
I'm back. I thought I had this script working but it seems to always stop
at the end of the C's, so I can get it to go thru pages A-C then it stops, I
have tried setting the time-out to a very large amount but it never
continues to run past the same spot, it gathers all records to the end of C,
then quits...any ideas?
Script:
http://www.greenecountymo.org/sheriff/warrants.php?search=A";;
$targets["b"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=B";;
$targets["c"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=C";;
$targets["d"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=D";;
$targets["e"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=E";;
$targets["f"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=F";;
$targets["g"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=G";;
$targets["h"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=H";;
$targets["i"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=I";;
$targets["j"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=J";;
$targets["k"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=K";;
$targets["l"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=L";;
$targets["m"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=M";;
$targets["n"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=N";;
$targets["o"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=O";;
$targets["p"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=P";;
$targets["q"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=Q";;
$targets["r"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=R";;
$targets["s"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=S";;
$targets["t"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=T";;
$targets["u"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=U";;
$targets["v"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=V";;
$targets["w"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=W";;
$targets["x"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=X";;
$targets["y"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=Y";;
$targets["z"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=Z";;
foreach
($targets as $target_url){
echo $target_url;
$userAgent = 'Googlebot/2.1
(http://www.googlebot.com/bot.html)';
$ch = curl_init();
curl_setopt($ch,
CURLOPT_USERAGENT, $userAgent);
curl_setopt($ch,
CURLOPT_URL,$target_url);
curl_setopt($ch, CURLOPT_FAILONERROR,
true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, false);
curl_setopt($ch,
CURLOPT_AUTOREFERER, true);
curl_setopt($ch,
CURLOPT_RETURNTRANSFER,true);
curl_setopt($ch, CURLOPT_TIMEOUT,
6000);
$html = curl_exec($ch);
if (!$html) {
echo "cURL error
number:" .curl_errno($ch);
echo "cURL error:" . curl_error($ch);
exit;
}
// Create DOM from URL or file
$html =
file_get_html($target_url);
// Find table
foreach($html->find('table') as
$table)
{
foreach($table->find('tr') as $tr)
{
// Grab children
$cells = $tr->children();
if($cells[0]->plaintext != "Name")
{
for ($i =
0; $i < count($cells); $i++)
{
switch ($i){
case 0: // Name
$name =
$cells[$i]->plaintext;
echo $cells[$i]->plaintext;
break;
case 1: // Age
$age =
$cells[$i]->plaintext;
break;
case 2:
// Warrant type
$warrant = $cells[$i]->plaintext;
break;
case 3: // Bond amount
$bond =
$cells[$i]->plaintext;
break;
case 4:
// Warrant number
$wnumber = $cells[$i]->plaintext;
break;
case 5: // Offence description
$crime = $cells[$i]->plaintext;
break;
Default:
echo "Uh-oh";
}
}
}
// Build your INSERT statement here
// Build your INSERT statement here
$query = "INSERT
into warrants (wid, name, age, warrant, bond, wnumber, crime) VALUES (";
$query .= " '$wid', '$name', '$age', '$warrant', '$bond', '$wnumber',
'$crime' )";
//$wid = mysql_insert_id();
echo
$query;
// run query
mysql_query($query) or die
