Re: Nutch crawls not appearing in Kibana

2015-06-22 Thread Lewis John Mcgibbney
Hi Jessica and Brooks, On Fri, Jun 19, 2015 at 10:06 AM, user-digest-h...@nutch.apache.org wrote: [snip] Notice the 'prevFetchTime' field has been updated to show the next date when this URL should be crawled (30 days from now - July 19). I assume this is exactly what SHOULD

Re: Nutch crawls not appearing in Kibana

2015-06-22 Thread Lewis John Mcgibbney
Actually please just see https://issues.apache.org/jira/browse/NUTCH-2045 If you guys could test it would be great. lewis On Mon, Jun 22, 2015 at 11:21 AM, Lewis John Mcgibbney lewis.mcgibb...@gmail.com wrote: Hi Jessica and Brooks, On Fri, Jun 19, 2015 at 10:06 AM,

Re: Nutch crawls not appearing in Kibana

2015-06-19 Thread Brooks Isoldi
Hello Lewis, Thank you for your response and I apologize for my delay. Below is my nutch-site.xml, note that I originally had ALL of the settings (uncommented) but started commenting them while trying to troubleshoot. https://gist.github.com/bisoldi/45631e9a011e39949cf2 So, I think we

Re: Nutch crawls not appearing in Kibana

2015-06-18 Thread Jessica Glover
I am having the same problem as Brooks, and I noticed that the timestamp field is one month ahead. This explains why you can't see the results in Kibana, but I'm wondering how to fix the timestamp problem. On Wed, Jun 17, 2015 at 3:20 PM, Lewis John Mcgibbney lewis.mcgibb...@gmail.com wrote:

Re: Nutch crawls not appearing in Kibana

2015-06-17 Thread Lewis John Mcgibbney
Hi Brooks On Wed, Jun 17, 2015 at 7:59 AM, user-digest-h...@nutch.apache.org wrote: Hi all, [snip] First things first, can you veryify your elastic search settings in nutch-site.xml? e.g. https://github.com/apache/nutch/blob/2.x/conf/nutch-default.xml#L1240-L1291 Please make sure that

Nutch crawls not appearing in Kibana

2015-06-16 Thread Brooks Isoldi
Hi all, FYI, I am also posting this on SO. I'm really not sure where the issue lies with this problem, whether it's a Nutch problem, Kibana or Elasticsearch. I'm using Nutch 2.3, HBase 0.94.14 and Elasticsearch 1.6 with Kibana 4.1.0 to crawl, archive and index. I primarily