Many Thanks. Ben ----- Original Message ----- From: "Matthew Walker" <[EMAIL PROTECTED]> To: "CF-Talk" <[email protected]> Sent: Tuesday, September 06, 2005 3:38 PM Subject: RE: CF Spider/Screen Scrape
> This may help, but you need to be able to use regular expressions.... > http://www.eswsoftware.com/library/scraping.cfm > > > -----Original Message----- > From: Matthew Blatchley [mailto:[EMAIL PROTECTED] > Sent: Tuesday, 6 September 2005 12:45 p.m. > To: CF-Talk > Subject: Re: CF Spider/Screen Scrape > > First you might want to start by having permission from the site owner > of > the page your scraping which isn't as easy as a phone call :) > > I would use <CFHTTP> and then search through the page for the specific > name > of the products your looking for and the price. It's a lot easier to > scrape > a page that isn't dynamically generated because it probably won't change > > from one day to the next. You'll need to use some pretty fancy Regular > expressions to get it work for each page you scrape. Once the data is > collected, send out the email with link to update as Matthew > described...but > again...scraping can be something you can get into trouble with so I'd > check > into seeing if your not doing something illegal. > > Matt > > > ----- Original Message ----- > From: "Nomad" <[EMAIL PROTECTED]> > To: "CF-Talk" <[email protected]> > Sent: Monday, September 05, 2005 7:05 PM > Subject: Re: CF Spider/Screen Scrape > > > > Thanks. > > How do I get started with the monitoring service? Any pointers? > > > > Ben > > > > ----- Original Message ----- > > From: "Matthew Walker" <[EMAIL PROTECTED]> > > To: "CF-Talk" <[email protected]> > > Sent: Tuesday, September 06, 2005 12:10 PM > > Subject: RE: CF Spider/Screen Scrape > > > > > >> Yes you can do this in CF, but... > >> > >> Screen scraping is very fragile. It can be useful, but I wouldn't > >> recommend attaching it to a live database. Your competitors could > change > >> their template and then suddenly you're selling your products for > $0.00. > >> Will you as the developer be liable? > >> > >> I would recommend, instead of changing the data automatically, just > >> compiling it into a mail message with links and letting the client > >> verify the prices by hand. So build a monitoring service. This would > be > >> a lot safer. > >> > >> Is what you want to do even legal? > >> > >> -----Original Message----- > >> From: Nomad [mailto:[EMAIL PROTECTED] > >> Sent: Tuesday, 6 September 2005 12:01 p.m. > >> To: CF-Talk > >> Subject: CF Spider/Screen Scrape > >> > >> Hello all, > >> > >> I have been asked by a client who operates an online shop to do the > >> following: > >> > >> 1. Create a spider/crawler that will crawl and collect pricing > >> information > >> from a number of competitors sites. > >> 2. Update the clients own database with the information collected > >> undercutting the competitors by few cents/dollars. > >> > >> Can Cold Fusion do this? > >> Can someone provide some pointers for me to get started. > >> I understand that this is not an easy beast to tame. > >> Is there any ready made solution that I can integrate with the > clients > >> website (CF based). > >> > >> Will appreciate any help. > >> > >> Thanks > >> > >> Ben Thomas. > >> > >> > >> > >> > >> > > > > > > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Find out how CFTicket can increase your company's customer support efficiency by 100% http://www.houseoffusion.com/banners/view.cfm?bannerid=49 Message: http://www.houseoffusion.com/lists.cfm/link=i:4:217508 Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4 Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4 Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4 Donations & Support: http://www.houseoffusion.com/tiny.cfm/54
