Michael Stack wrote:
The javascript parser will often add the discovered URL as its anchor
text (See below linkdb dump for examples). These urls-as-anchor text
are tokenized when indexing and then, because anchors by default get a
hefty boost at query time, the URL-found-by-the-parse-js-plugin can
show high in search results.
Is adding the URL as anchor intentional? To me it looks like anchor
text pollution (or, if not, to be consistent, anytime there is empty
anchor text, we should just add inlink URL).
When I initially wrote this plugin I thought that providing at least
some anchor text is better than no text at all - but now I don't think
so anymore, exactly because of this reason ... So I agree, we should
just put empty String if there's no anchor.
Also, while on the parse-js, I see a lot of ERROR-level logging
complaining of malformed URLs (See below for example); the matches
from a regex over javascript content are being passed to java.net.URL
for it to figure whether the javascript substring is a likely URL or
not. Should these messages be logged instead at INFO level (Or the
regex tightened up so more likely the string passed is actually an URL).
I'll make patches dependent on feedback.
Any improvements here to extract more likely URLs are welcome. I'd
rather see them as additional hardcoded rules than an expanded regex, if
it makes sense - complex regexes often misbehave on long texts, either
hanging or slowing down and consuming 100% CPU.
--
Best regards,
Andrzej Bialecki <><
___. ___ ___ ___ _ _ __________________________________
[__ || __|__/|__||\/| Information Retrieval, Semantic Web
___|||__|| \| || | Embedded Unix, System Integration
http://www.sigram.com Contact: info at sigram dot com
