First you might want to start by having permission from the site owner of the page your scraping which isn't as easy as a phone call :)
I would use <CFHTTP> and then search through the page for the specific name of the products your looking for and the price. It's a lot easier to scrape a page that isn't dynamically generated because it probably won't change from one day to the next. You'll need to use some pretty fancy Regular expressions to get it work for each page you scrape. Once the data is collected, send out the email with link to update as Matthew described...but again...scraping can be something you can get into trouble with so I'd check into seeing if your not doing something illegal. Matt ----- Original Message ----- From: "Nomad" <[EMAIL PROTECTED]> To: "CF-Talk" <[email protected]> Sent: Monday, September 05, 2005 7:05 PM Subject: Re: CF Spider/Screen Scrape > Thanks. > How do I get started with the monitoring service? Any pointers? > > Ben > > ----- Original Message ----- > From: "Matthew Walker" <[EMAIL PROTECTED]> > To: "CF-Talk" <[email protected]> > Sent: Tuesday, September 06, 2005 12:10 PM > Subject: RE: CF Spider/Screen Scrape > > >> Yes you can do this in CF, but... >> >> Screen scraping is very fragile. It can be useful, but I wouldn't >> recommend attaching it to a live database. Your competitors could change >> their template and then suddenly you're selling your products for $0.00. >> Will you as the developer be liable? >> >> I would recommend, instead of changing the data automatically, just >> compiling it into a mail message with links and letting the client >> verify the prices by hand. So build a monitoring service. This would be >> a lot safer. >> >> Is what you want to do even legal? >> >> -----Original Message----- >> From: Nomad [mailto:[EMAIL PROTECTED] >> Sent: Tuesday, 6 September 2005 12:01 p.m. >> To: CF-Talk >> Subject: CF Spider/Screen Scrape >> >> Hello all, >> >> I have been asked by a client who operates an online shop to do the >> following: >> >> 1. Create a spider/crawler that will crawl and collect pricing >> information >> from a number of competitors sites. >> 2. Update the clients own database with the information collected >> undercutting the competitors by few cents/dollars. >> >> Can Cold Fusion do this? >> Can someone provide some pointers for me to get started. >> I understand that this is not an easy beast to tame. >> Is there any ready made solution that I can integrate with the clients >> website (CF based). >> >> Will appreciate any help. >> >> Thanks >> >> Ben Thomas. >> >> >> >> >> > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Logware (www.logware.us): a new and convenient web-based time tracking application. Start tracking and documenting hours spent on a project or with a client with Logware today. Try it for free with a 15 day trial account. http://www.houseoffusion.com/banners/view.cfm?bannerid=67 Message: http://www.houseoffusion.com/lists.cfm/link=i:4:217407 Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4 Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4 Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4 Donations & Support: http://www.houseoffusion.com/tiny.cfm/54
