I'm writing a quick personal app that will mine a few websites on an hourly basis to grab various bits of data and file them away in a DB for later analysis.
The two simplest technologies I can use to emulate a browser and grab some basic html from a website are jwebunit/httpunit and Curl. Does anyone have any advice or reccomendations when it comes to writing a simple data-mining application for the web using either of these? (I am particularly interested in a recommendation between jwebunit (which I like because it's java based, even if it's more a testing framework than a browser emulator) and curl (which is just a simple command line tool that can grab data from a site for later parsing.) Has anyone done anything like this using jwebunit or another technology before? (Doubtless they have :) I could really use some advice before I continue though.) Thanks, - alasdair Alasdair Young -- No virus found in this outgoing message. Checked by AVG Free Edition. Version: 7.1.375 / Virus Database: 268.2.2/280 - Release Date: 03/13/2006 ------------------------------------------------------- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642 _______________________________________________ Jwebunit-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/jwebunit-users
