Hi all,
I want to download the top of an html file, parse it for searching a string,
and if that string is found, I want to cancel downloading the rest of the
file.
If the string is not found, download the second part, and so on.
The problem is that even though I've set the chunk size to 6500, some chunks
have only a few hundread bytes, and this could be a problem because it might
split the string and it won't be found.
Can you tell me if it is a problem with my script, or with LWP?
Thank you very much!
#!/usr/bin/perl
use strict;
use LWP::UserAgent;
my $out = "/var/www/teddy/writable/e1.txt";
my $ua = LWP::UserAgent->new(env_proxy => 0,
timeout => 40,
keep_alive => 1
);
$ua -> agent('Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)');
my $location = 'http://.../';
my $min = 0;
my $max = 100;
ADDR: for (my $i = $min;$i < $max; $i++) {
my $request = HTTP::Request->new('GET', $location);
$request -> header('REFERER' => $location);
my $response = $ua->request($request, \&callback, 6500);
}
sub callback {
my($data, $response, $protocol) = @_;
my @a = $data =~ /$string/i;
if ($a[0]) {
open (OUT, ">>$out") or die "Can't write to $out - $!";
print OUT $a[0], "\n";
close OUT;
next ADDR;
}
}
Thank you.
Teddy,
Teddy's Center: http://teddy.fcc.ro/
Email: [EMAIL PROTECTED]
