> That's why I'm confused. I've done complex screen scraping on > ebay and others and it never takes more than a few seconds > per request, even big ones. Maybe I write tighter regex, but
This is true, in fact it is only taking about 1 second per item to gather in the page and regex it for the prices, however it is doing 7500+ products and some simple maths will tell you that 7500 seconds is 125 minutes, which is just over 2 hours. > the point is that it should never take hours. In addition, > most places that you'd want data from have one form or > another of web services which can speed everything up even more. These are being grabbed from Amazon, and they have assured me the only access to the data is via the individual product pages. -- Jay ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Discover CFTicket - The leading ColdFusion Help Desk and Trouble Ticket application http://www.houseoffusion.com/banners/view.cfm?bannerid=48 Message: http://www.houseoffusion.com/lists.cfm/link=i:5:163040 Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/5 Subscription: http://www.houseoffusion.com/lists.cfm/link=s:5 Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.5 Donations & Support: http://www.houseoffusion.com/tiny.cfm/54
