"Beyond Control Inc." <[EMAIL PROTECTED]> writes:
> Unfortunatly, if I start it now it *will* work for 10 hours, then die.
How many pages has been fetched at this point? 6 each hour for 12
hours are 72, right?
Does it fail quickly if you don't wait for 10 minutes between each
> This is why I find this to be so weird, since I look like IE5, and I always
> create a new UserAgent object so that I won't get cookied or anything like
> ----- Original Message -----
> From: "William R Ward" <[EMAIL PROTECTED]>
> To: <[EMAIL PROTECTED]>
> Sent: Wednesday, November 28, 2001 11:55 PM
> Subject: Re: 500 Read error
> > [EMAIL PROTECTED] (Beyond Control Inc.) writes:
> > > I've written a small script that fetches a page every 10 minutes.
> > >
> > > After 12-14 hours, it suddenly stops fetching the page, say there's a
> > > read error (500)
> > Is this something that works once you start it up again? Or did it
> > work for a while and now you can't get it to work anymore? If the
> > latter, it could be that the web administrators noticed your script
> > and blocked it. Some web site owners don't like people slurping their
> > data when the ads or copyright notices might not be slurped as well.