Have you tried using wget or curl instead of lynx? Both were created for spidering so they presumably have options for specifying different output files (or the command line). The unix man pages for both contain good info on the various arguments.
AFAIK both have win32 builds, I think I even have them on my machine at work. Chances are a google search will help you find them but I can email them to you on Wednesday if you like.
Cheers, Vik
Mark Stanton wrote:
Hi All
Just a couple of quick questions that I was struggling with last night, maybe someone can help me. I'm trying to set up a spider using lynx (with -traversal -crawl) on WIN2K, some of my issues are specifically to do with this others are more general.
The two most general issues are: - cfexecute seems to run for exactly the timeout period regardless of how long a process actually takes place. Is this correct? - What directory does cfexecute work in when it is run and can this be changed? From what I can see it seems to run from the directory in which jrun lives. My problem here is that lynx just dumps all its logs & cached pages in the current directory and there is no provision in lynx to specify an alternate location.
Lynx specific question: - Has anyone played with lynx -traversal before especially in conjunction with coldfusion? The biggest problem that I am having is that when I run the command from coldfusion (via cfexecute) there don't seem to be ANY files generated although I've got it working manually from the command line (so I know my syntax is good) and lynx is definitely being run.
I'm a bit confused so any help is appreciated. I can hear Geoff already saying "Verity Spider" :)
Bit more background, I'm trying to set up a method of following all links within a domain, reading all pages and checking each one for validity & mark up errors using tidy. Other side things that will easily spin off from this are having all the text content on the site downloaded for easy spell checking & also automatic dead link checking.
I've got all the pieces working separately, but as soon as I try & get them together under CF I trip over the first hurdle :(
Cheers
Mark
______________ Mark Stanton Web Production Gruden Pty Ltd Tel: 9956 6388 Mob: 0410 458 201 Fax: 9956 8433 www.gruden.com
******************************************************************** This message and any attachments are confidential to the ordinary user of the e-mail address to which it was addressed and may also be privileged. If you are not the addressee you may not copy, forward, disclose or use any part of the message or its attachments and if you have received this message in error, please notify the sender immediately by return e-mail and delete it from your system.
Internet communications cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, arrive late or contain viruses. The sender therefore does not accept liability for any errors or omissions in the context of this message which arise as a result of Internet transmission.
Before opening or using attachments, check them for viruses and defects. Our liability is limited to resupplying any affected attachments. ********************************************************************
--- You are currently subscribed to cfaussie as: [EMAIL PROTECTED] To unsubscribe send a blank email to [EMAIL PROTECTED]
MX Downunder AsiaPac DevCon - http://mxdu.com/
