Thanks for your replies to this - it helped locate the problem - the hosts file was pointing to a different ip so when the machine looked itself up it was getting a different ip then is specified in the DNS.
Thanks much, Josh --- Gilles Detillieux <[EMAIL PROTECTED]> wrote: > According to olinux: > > I brought this up a few weeks back and still have > not > > found a solution. when i set start_url to > > http://www.domain.com htdig immediately returns a > 404 > > error and does not index anyhting. I thought this > may > > have something to do with the DNS configuration, > but > > changes (adding a record for www.domain.com) have > had > > no effect. My tech support tells me that the > indexer > > is to blame. I doubt this, but don't know where > else > > to look. > > > > everything runs smooth when I use start_url: > > http://domain.com > > Yes, this was on Sept. 20, and back then I asked... > > >> Why is htdig 3.1.6 sending an HTTP/1.1 request to > your server??? The > >> 3.1.x versions of htdig only support HTTP/1.0? > What changes or patches > >> have you made to your htdig source? What version > did you really start > >> off with? > > I never did get an answer to that. You claimed that > both the domain.com > and www.domain.com domains work fine from a browser, > but were you > running this browser on the very same server from > which you run htdig and > your web server? (Most linux distributions include > the lynx text-based > browser, which is good for tests like this.) If the > browser doesn't have > problems with either domain, then it would seem > htdig is the culprit, > and it certainly seems suspicious to me that it > would be sending > "GET /index.htm HTTP/1.1" commands. Or was that > just a trascription > error on your part 2 weeks ago? If htdig was > patched, it would be helpful > to know what the changes were, and whether it does > reliably implement the > full HTTP/1.1 protocol (which vanilla 3.1.6 > definitely doesn't!). If it's > really sending HTTP/1.0 requests, not 1.1, then I'd > be puzzled to find out > that lynx on the same machine doesn't have a problem > with the same URL. > > Does your web server implement different virtual > server configurations for > domain.com and www.domain.com, or are the two > completely equivalent? Even > if your htdig was hacked with a half-baked HTTP/1.1 > implementation, it still > seems odd that it would work for one domain but not > the other. > > -- > Gilles R. Detillieux E-mail: > <[EMAIL PROTECTED]> > Spinal Cord Research Centre WWW: > http://www.scrc.umanitoba.ca/ > Dept. Physiology, U. of Manitoba Winnipeg, MB R3E > 3J7 (Canada) > > > ------------------------------------------------------- > This sf.net email is sponsored by:ThinkGeek > Welcome to geek heaven. > http://thinkgeek.com/sf > _______________________________________________ > htdig-general mailing list > <[EMAIL PROTECTED]> > To unsubscribe, send a message to > <[EMAIL PROTECTED]> with a > subject of unsubscribe > FAQ: http://htdig.sourceforge.net/FAQ.html __________________________________________________ Do you Yahoo!? Faith Hill - Exclusive Performances, Videos & More http://faith.yahoo.com ------------------------------------------------------- This sf.net email is sponsored by:ThinkGeek Welcome to geek heaven. http://thinkgeek.com/sf _______________________________________________ htdig-general mailing list <[EMAIL PROTECTED]> To unsubscribe, send a message to <[EMAIL PROTECTED]> with a subject of unsubscribe FAQ: http://htdig.sourceforge.net/FAQ.html
