> At my job we have a secure website. Every hit to the site is captured by the > tracking > system to the SQL Server database. > > We need to create an inventory system that can look at the data and tell us > about > the assets on the site. > > To get the appropriate data into the database, we need to use a directory > crawler that > can hit every asset and every item that it finds in the directory structure. > > Is there such a crawler, that can appear to be a user-agent, that can crawl a > secure > website? Is there such a crawler in ColdFusion?
If you want to do this using CF, you can use the CFHTTP tag as Michael mentioned. But it's not clear to me that you're capturing the information on each page; if you just need to make an HTTP request to each page so that the existing logging system logs visits, there are far easier approaches, such as wget. Of course, crawlers typically only follow plain ol' HTML links, so if you have forms-driven navigation or JavaScript-driven navigation you'll have to figure out how to get to everything. Dave Watts, CTO, Fig Leaf Software http://www.figleaf.com/ Fig Leaf Software provides the highest caliber vendor-authorized instruction at our training centers in Washington DC, Atlanta, Chicago, Baltimore, Northern Virginia, or on-site at your location. Visit http://training.figleaf.com/ for more information! ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Want to reach the ColdFusion community with something they want? Let them know on the House of Fusion mailing lists Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:324332 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
