Re: AW: AW: [htdig] add parameter to the url while indexing

[Gilles Detillieux]

According to Thieme, Winfried:
> I already tried the url_rewrite_rules, but i got a strange 
> behavior.
> 
> E.g. the rule (.*)\\?(.*) \\0\\&param=value should append
> my parameter to every url with an already existing parameter.
> But the spider indexes infinitely in a recursive manner:
> 
>       url: abc.com/test?test=something
>       
>       -> abc.com/test?test=something&param=value
>       -> abc.com/test?test=something&param=value&param=value
>       -> abc.com/test?test=something&param=value&param=value&param=value
>       -> ...

It seems that somehow the same URL is being fed back into the queue, and so
the rewriting keeps adding another parameter to the same URL, making it a
different URL.  You might be able to add another rule to get rid of the
parameter before it's added back on, so it's never added more than once.
E.g.:

url_rewrite_rules:      (.*\\?.*)&param=value\$ \\1 \
                        (.*)\\?(.*) \\0\\&param=value

-- 
Gilles R. Detillieux              E-mail: <[EMAIL PROTECTED]>
Spinal Cord Research Centre       WWW:    http://www.scrc.umanitoba.ca/
Dept. Physiology, U. of Manitoba  Winnipeg, MB  R3E 3J7  (Canada)

_______________________________________________
htdig-general mailing list <[EMAIL PROTECTED]>
To unsubscribe, send a message to <[EMAIL PROTECTED]> with a 
subject of unsubscribe
FAQ: http://htdig.sourceforge.net/FAQ.html