[
https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16072959#comment-16072959
]
Lewis John McGibbney commented on NUTCH-1465:
---------------------------------------------
[~markus17] when attempting to process the following sitemap -
http://www.autotrader.com/sitemap.xml, it appears the new processor is not able
to process anything... although the crawldb data structures are produced, no
entries are added... can you please rescope the patch and ensure it is the most
up-to-date one you are working with? Thanks
{code}
2017-07-03 15:32:09,213 INFO util.SitemapProcessor - SitemapProcessor: Total
records rejected by filters: 0
2017-07-03 15:32:09,213 INFO util.SitemapProcessor - SitemapProcessor: Total
sitemaps from HostDb: 0
2017-07-03 15:32:09,213 INFO util.SitemapProcessor - SitemapProcessor: Total
sitemaps from seed urls: 1
2017-07-03 15:32:09,213 INFO util.SitemapProcessor - SitemapProcessor: Total
failed sitemap fetches: 0
2017-07-03 15:32:09,213 INFO util.SitemapProcessor - SitemapProcessor: Total
new sitemap entries added: 0
2017-07-03 15:32:09,213 INFO util.SitemapProcessor - SitemapProcessor:
Finished at 2017-07-03 15:32:09, elapsed: 00:00:19
{code}
> Support sitemaps in Nutch
> -------------------------
>
> Key: NUTCH-1465
> URL: https://issues.apache.org/jira/browse/NUTCH-1465
> Project: Nutch
> Issue Type: New Feature
> Components: parser
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Fix For: 1.14
>
> Attachments: NUTCH-1465.patch, NUTCH-1465.patch, NUTCH-1465.patch,
> NUTCH-1465-sitemapinjector-trunk-v1.patch, NUTCH-1465-trunk.v1.patch,
> NUTCH-1465-trunk.v2.patch, NUTCH-1465-trunk.v3.patch,
> NUTCH-1465-trunk.v4.patch, NUTCH-1465-trunk.v5.patch
>
>
> I recently came across this rather stagnant codebase[0] which is ASL v2.0
> licensed and appears to have been used successfully to parse sitemaps as per
> the discussion here[1].
> [0] http://sourceforge.net/projects/sitemap-parser/
> [1]
> http://lucene.472066.n3.nabble.com/Support-for-Sitemap-Protocol-and-Canonical-URLs-td630060.html
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
