[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394940#comment-16394940 ] Sebastian Nagel commented on NUTCH-1465: The feature is already ported to 2.x, see NUTCH-1741, but

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393274#comment-16393274 ] Ben Vachon commented on NUTCH-1465: --- Is there any plan to pull this to 2.x? > Support sitemaps in Nutch

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16209562#comment-16209562 ] ASF GitHub Bot commented on NUTCH-1465: --- sebastian-nagel commented on issue #189: NUTCH-1465 Support

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16209547#comment-16209547 ] ASF GitHub Bot commented on NUTCH-1465: --- marconett commented on issue #189: NUTCH-1465 Support

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127556#comment-16127556 ] ASF GitHub Bot commented on NUTCH-1465: --- sebastian-nagel closed pull request #202: NUTCH-1465

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127555#comment-16127555 ] ASF GitHub Bot commented on NUTCH-1465: --- sebastian-nagel commented on issue #202: NUTCH-1465 Support

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127483#comment-16127483 ] ASF GitHub Bot commented on NUTCH-1465: --- lewismc commented on issue #202: NUTCH-1465 Support for

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16101785#comment-16101785 ] ASF GitHub Bot commented on NUTCH-1465: --- lewismc closed pull request #195: NUTCH-1465 Support

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16101783#comment-16101783 ] ASF GitHub Bot commented on NUTCH-1465: --- lewismc commented on issue #202: NUTCH-1465 Support for

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16101784#comment-16101784 ] ASF GitHub Bot commented on NUTCH-1465: --- lewismc closed pull request #189: NUTCH-1465 Support

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16092964#comment-16092964 ] Hudson commented on NUTCH-1465: --- SUCCESS: Integrated in Jenkins build Nutch-trunk #3435 (See

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16092945#comment-16092945 ] Markus Jelsma commented on NUTCH-1465: -- Crap! I was probably looking without seeing! Got it! >

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16092938#comment-16092938 ] Sebastian Nagel commented on NUTCH-1465: I've modified the description of the properties

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16092909#comment-16092909 ] Markus Jelsma commented on NUTCH-1465: -- Sebastian, your patch has CrawlDatum and

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16090460#comment-16090460 ] Markus Jelsma commented on NUTCH-1465: -- Thanks! Will grab 202.patch and see if it fits tomorrow! >

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16090044#comment-16090044 ] ASF GitHub Bot commented on NUTCH-1465: --- sebastian-nagel opened a new pull request #202: NUTCH-1465

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16090038#comment-16090038 ] Sebastian Nagel commented on NUTCH-1465: Thanks, [~markus.jel...@openindex.io]! Tested on a small

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16078124#comment-16078124 ] Markus Jelsma commented on NUTCH-1465: -- I think this is committable, anyone to disagree? If not, i'll

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16074554#comment-16074554 ] Markus Jelsma commented on NUTCH-1465: -- Hi Lewis, 0.8 doesn't deal with this sitemap at autotrader

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16074046#comment-16074046 ] Lewis John McGibbney commented on NUTCH-1465: - [~markus17] can we also update the version of

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16073338#comment-16073338 ] Markus Jelsma commented on NUTCH-1465: -- Ah, i see. The autotrader sitemap points to an index of

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16073288#comment-16073288 ] Markus Jelsma commented on NUTCH-1465: -- Hello Lewis, I am positive i took the latest pieces. And

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16072959#comment-16072959 ] Lewis John McGibbney commented on NUTCH-1465: - [~markus17] when attempting to process the

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16072953#comment-16072953 ] ASF GitHub Bot commented on NUTCH-1465: --- lewismc opened a new pull request #195: NUTCH-1465 Support

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16070485#comment-16070485 ] Markus Jelsma commented on NUTCH-1465: -- Hi Lewis! It appears to be working fine now and bug-free due

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16070468#comment-16070468 ] Lewis John McGibbney commented on NUTCH-1465: - Fantastic [~markus17] is this working well for

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16070314#comment-16070314 ] Markus Jelsma commented on NUTCH-1465: -- There is an oddity going on when a sitemap.xml entry is

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16016943#comment-16016943 ] ASF GitHub Bot commented on NUTCH-1465: --- lewismc commented on issue #189: NUTCH-1465 Support

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16016914#comment-16016914 ] ASF GitHub Bot commented on NUTCH-1465: --- lewismc commented on a change in pull request #189:

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15986615#comment-15986615 ] ASF GitHub Bot commented on NUTCH-1465: --- sebastian-nagel commented on a change in pull request #189:

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15986614#comment-15986614 ] ASF GitHub Bot commented on NUTCH-1465: --- sebastian-nagel commented on a change in pull request #189:

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15986617#comment-15986617 ] ASF GitHub Bot commented on NUTCH-1465: --- sebastian-nagel commented on a change in pull request #189:

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15986618#comment-15986618 ] ASF GitHub Bot commented on NUTCH-1465: --- sebastian-nagel commented on a change in pull request #189:

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15986616#comment-15986616 ] ASF GitHub Bot commented on NUTCH-1465: --- sebastian-nagel commented on a change in pull request #189:

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15985680#comment-15985680 ] ASF GitHub Bot commented on NUTCH-1465: --- lewismc commented on issue #189: NUTCH-1465 Support

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15985678#comment-15985678 ] ASF GitHub Bot commented on NUTCH-1465: --- lewismc commented on a change in pull request #189:

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15985676#comment-15985676 ] ASF GitHub Bot commented on NUTCH-1465: --- lewismc commented on a change in pull request #189:

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15985671#comment-15985671 ] ASF GitHub Bot commented on NUTCH-1465: --- lewismc opened a new pull request #189: NUTCH-1465 Support

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15979161#comment-15979161 ] Sebastian Nagel commented on NUTCH-1465: Hi Lewis, a couple of month ago I've applied the latest

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15979078#comment-15979078 ] Lewis John McGibbney commented on NUTCH-1465: - I'm going to take this on. We want full sitemap

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13887588#comment-13887588 ] Sebastian Nagel commented on NUTCH-1465: filters and normalizers: -noFilter is not

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13887763#comment-13887763 ] Tejas Patil commented on NUTCH-1465: Re filters and normalizers: +1. Re fetch

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13888220#comment-13888220 ] Sebastian Nagel commented on NUTCH-1465: ??(1) fetch interval: ...?? +1, sounds

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13886453#comment-13886453 ] Sebastian Nagel commented on NUTCH-1465: Thanks, [~tejasp] for the improvements!

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13886489#comment-13886489 ] Sebastian Nagel commented on NUTCH-1465: SitemapReducer overwrites score, modified

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13886677#comment-13886677 ] Tejas Patil commented on NUTCH-1465: Interesting comments [~wastl-nagel]. Re filters

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882968#comment-13882968 ] Sebastian Nagel commented on NUTCH-1465: Great, looks good and is a really compact

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13883204#comment-13883204 ] Tejas Patil commented on NUTCH-1465: Hi [~wastl-nagel], Thanks a lot for your

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13883523#comment-13883523 ] Sebastian Nagel commented on NUTCH-1465: Sorry, you're right: the comment hacky

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13879955#comment-13879955 ] Lewis John McGibbney commented on NUTCH-1465: - Hey [~tejasp]. Again, great

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13880295#comment-13880295 ] Tejas Patil commented on NUTCH-1465: Hi [~lewismc], +1 for the first two suggestions.

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13880305#comment-13880305 ] Lewis John McGibbney commented on NUTCH-1465: - hey [~tejasp] no probs. RE: #3,

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13862237#comment-13862237 ] Tejas Patil commented on NUTCH-1465: Hi [~wastl-nagel], Yes. I think that it should be

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13848998#comment-13848998 ] Sebastian Nagel commented on NUTCH-1465: Let's add use case C: *(C) inject URLs

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13848561#comment-13848561 ] Tejas Patil commented on NUTCH-1465: Revisited this Jira after a long time and gave a

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13848723#comment-13848723 ] Tejas Patil commented on NUTCH-1465: Hi [~wastl-nagel], Nice share. The only grudge I

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13721045#comment-13721045 ] Brian commented on NUTCH-1465: -- Is a separate issue needed for support in 2.X?

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13564274#comment-13564274 ] Sebastian Nagel commented on NUTCH-1465: Hi Tejas, thanks and a few comments on

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13564768#comment-13564768 ] Sebastian Nagel commented on NUTCH-1465: Yes, SitemapInjector is a map-reduce

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13564836#comment-13564836 ] Markus Jelsma commented on NUTCH-1465: -- Thanks all for your interesting comments.

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13564883#comment-13564883 ] Tejas Patil commented on NUTCH-1465: Hi Sebastian, So we are looking at 2 things

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13564019#comment-13564019 ] Ken Krugler commented on NUTCH-1465: Hi Tejas - I thought the current CC robots

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13564040#comment-13564040 ] Tejas Patil commented on NUTCH-1465: Hi Ken, As the CC robots integration jira is not

[jira] [Commented] (NUTCH-1465) Support sitemaps in Nutch

[ https://issues.apache.org/jira/browse/NUTCH-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13564057#comment-13564057 ] Ken Krugler commented on NUTCH-1465: Hi Tejas - the original code didn't, but I