[
https://issues.apache.org/jira/browse/NUTCH-1661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13833459#comment-13833459
]
Otis Gospodnetic commented on NUTCH-1661:
-----------------------------------------
Can you describe this a bit more? So this assumes the page was fetched once,
its language was identified and stored, and the next time Nutch considers
fetching this page the page may be rejected/excluded from fetching based on its
language?
There is a loop in the code that may be a little nicer/faster if converted to a
Set lookup.
> Language based crawling
> -----------------------
>
> Key: NUTCH-1661
> URL: https://issues.apache.org/jira/browse/NUTCH-1661
> Project: Nutch
> Issue Type: Improvement
> Affects Versions: 2.2.1
> Reporter: Talat UYARER
> Priority: Minor
> Labels: PatchAvailable
> Fix For: 2.3
>
> Attachments: NUTCH-1661.patch
>
>
> If you want to crawling based on specific language, Nutch has not any
> solution. I created a custom Fetch Scheduler. It controls page's language
--
This message was sent by Atlassian JIRA
(v6.1#6144)
