Greetings,
I am indexing a site which contains a lot of complex hrefs like:
<a href="/index2.html?wh&top/whatsnew_en.html">
When I run with -vvv and grep "push" to see what it is indexing:
with 3.1.5, I get:
pushing http://author82/index.html
pushing http://author82/index2.html?wh&top/whatsnew_en.html
etc...
and the whole site is successfully indexed.
with 3.1.6, I get:
pushing http://author82/index.html
end of story...
only the top (plain) documents are indexed and htdig does not push any
of the complex URLs onto its stack.
I tried setting "max_description_length: 256" but no effect. I suspect
there is something in 3.1.6 which causes htdig not to recognise the
complex URLs which contain "?" and "&" but I can't find any directive
like "allow_in_url".
Note that I do not see any "rejected" comments in the trace.
Does anyone know how I can activate these URLs in 3.1.6?
Rgds,
Owen Boyle.
_______________________________________________
htdig-general mailing list <[EMAIL PROTECTED]>
To unsubscribe, send a message to <[EMAIL PROTECTED]> with a
subject of unsubscribe
FAQ: http://htdig.sourceforge.net/FAQ.html
