> I have a script that takes a list of URL's into an array
> and gets the page, strips the anchors to an array, cleans
> it up and matches the contents of each page against a
> search variable, printing the hits to a web page.
Are these always the same URLs?
<TEACHER SPEAK>
don't use a ' unless it's a contraction.
</TEACHER SPEAK>
If so save the output to a file and you don't have to process every time.
Do you know in advance which URLs will be processed? If so use a cron
job to process ahead of time.
> This works well with a smaller list of up to 12 or so URL's.
> After that it slows down way out of proportion to the number
> of URL's in the list. It also spits it back in 8K chunks. Any
> ideas?
Sound like what you have here is what we propeller heads call an N
square algorithm. The amount of time to process increases by the square
of the number of items. Most common sorts are N square. The solution is
trying to develope a different algorithm to do the job, not refine the
existing algorithm.
> How do I run multiple sockets at the same time? I don't know
> how to leave one and start another. The Programming Perl book
> does not say much about it. I'd like to use one to go through
> the primary array creating secondary arrays and spawning a socket
> for each secondary array. Is this possible?
As the old saw says, "Anything can be done, the impossible just
takes a little longer." fork creates a second -- and concurrent --
process. However, programming and synchronizing is non-trivial.
Difficult to comment without seeing the code. I'd be happy to take
a look and offer some suggestions if you would send it to me.
Urb, [EMAIL PROTECTED]
Visit America's Town Square. Creators of extraordinary Web sites.
http://www.usats.com Build your own Home Page without knowing
any HTML. Many user selectable options. After creation, your page
is e-mailed to you. http://usats.com/homepage
____________________________________________________________________
--------------------------------------------------------------------
Join The Web Consultants Association : Register on our web site Now
Web Consultants Web Site : http://just4u.com/webconsultants
If you lose the instructions All subscription/unsubscribing can be done
directly from our website for all our lists.
---------------------------------------------------------------------
