RE: Remotely spider websites

2003-02-18 Thread Dave Watts
> > From what I have been able to find, the verity spider will > > only spider sites on a single network domain... so a remote > > machine must reside on the same domain. What I'm wondering > > is if anybody > > Yup. Verity (and MS Index Server) are what's called "Worms" - > they "burrow" t

RE: Remotely spider websites

2003-02-06 Thread Jillian Carroll
ilto:[EMAIL PROTECTED]] > Sent: February 5, 2003 9:45 AM > To: CF-Talk > Subject: RE: Remotely spider websites > > > > -Original Message- > > From: Jillian Carroll [mailto:[EMAIL PROTECTED]] > > Sent: Wednesday, February 05, 2003 10:18 AM > > To: CF-Talk &g

RE: Remotely spider websites

2003-02-05 Thread Everett, Al
Let me add: Please, for the love of Pete, respect the directives in the robots.txt file. > -Original Message- > From: Jim Davis [mailto:[EMAIL PROTECTED]] > Sent: Wednesday, February 05, 2003 10:45 AM > To: CF-Talk > Subject: RE: Remotely spider websites > > >

RE: Remotely spider websites

2003-02-05 Thread Jim Davis
> -Original Message- > From: Jillian Carroll [mailto:[EMAIL PROTECTED]] > Sent: Wednesday, February 05, 2003 10:18 AM > To: CF-Talk > Subject: Remotely spider websites > > > >From what I have been able to find, the verity spider will > only spider >

Remotely spider websites

2003-02-05 Thread Jillian Carroll
>From what I have been able to find, the verity spider will only spider sites on a single network domain... so a remote machine must reside on the same domain. What I'm wondering is if anybody can recommend (or has written) a tool that will let me spider remote sites in a CF-friendly manner? Tha